Techniques to detect poisoning of an automatic speech recognition model

Abstract

The present invention proposes techniques to detect poisoning of an Automatic speech recognition model. A system (10) detects poisoning of an Audio speech recognition (ASR) model. The ASR model (2) adapted to generate a transcript from an input audio signal. A processor (1) is configured to: feed a clean-input to an autoencoder (3) to generate a reconstructed clean-input; feed the clean-input to an ASR model (2) to obtain a transcript; feed the reconstructed clean-input to the ASR model(2) to obtain a reconstructed transcript; compare the transcript with the reconstructed transcript to determine a pre-defined threshold, and detect poisoning of the ASR model (2) based on the pre-defined threshold.

Description

FORM 2THE PATENTS ACT, 1970 (39 of 1970) & the Patents Rules 2003COMPLETE SPECIFICATION (SECTION 10 and Rule 13)1. Title of the Invention:Techniques to detect poisoning of an Automatic speech recognition model.2. Applicants: a. Name: Bosch Global Software Technologies Private Limited. Nationality: INDIAAddress: 123, Industrial Layout, Hosur Road, Koramangala, Bangalore - 560095, Karnataka, India b. Name: Robert Bosch GmbHNationality: GERMANYAddress: Feuerbach, Stuttgart, GermanyComplete Specification:The following specification describes and ascertains the nature of this invention and the manner in which it is to be performedField of the invention

[0001] The present disclosure relates to the field of Artificial Intelligence security. In particular, it proposes a method to detect poisoning of an automatic speech recognition model.Background of the invention

[0002] ASR (automatic speech recognition) is an artificial intelligence technology wherein a machine learns to recognize a spoken language into text. ASR systems are used in various applications such as voice assistants, transcription services, and interactive voice response (IVR) systems.

[0003] Most of the Al based systems, receive large amounts of data and process the data to train Al models. Trained Al models generate output based on the use cases requested by the user. It is possible that some adversary may try to tamper / manipulate / evade the model in Al Systems to create incorrect outputs. The adversary may use different techniques to manipulate the output of the model. One of the simplest techniques used is wherein the adversary may manipulate the input data to bring an artificial output. The same scheme is typically phrased as ‘data poisoning’ and is defined as an effective attack against machine learning models that threatens model integrity by introducing poisoned data into the training dataset. Since the model learns on poisoned dataset, it is bound to give incorrect results. In many applications of machine learning, such as authentication systems, the training data is non- stationary. Both the joining of new users and the leaving of old users will affect the distribution of the whole dataset. In order to handle such non- stationary data distribution, the model typically needs to be retrained periodically. When the models are retrained on new samples collected during network operation, it gives the attacker a chance to inject poisoned samples into the training dataset. Specifically, the attacker may manipulate the training sample.

[0004] This will cause hardships to the original developer of the Al in the form of business disadvantages, loss of confidential information, loss of lead time spent in development, loss of intellectual properties, loss of future revenues etc. Data poisoning can render machine learning models inaccurate, possibly resulting in poor decisions based on faulty outputs.

[0005] While the existing prior arts suggest defense mechanisms for adversarial data poisoning for machine learning systems such as image classifiers, defense against data poisoning attacks on speaker recognition systems are not much studied.

[0006] The prior art “Ke Li, Cameron Baird, Dan Lin : Defend Data Poisoning Attacks on Voice Authentication1" proposed an easy-to-implement data poisoning attack to the voice authentication system, which can hardly be captured by existing defense mechanisms and then the prior art proposes a more robust defense method, called Guardian, which is a convolutional neural network-based discriminator. The Guardian discriminator integrates a series of novel techniques including bias reduction, input augmentation, and ensemble learning.

[0007] The present disclosure proposes to detect data poisoning in ASR systems. The objective is to detect any targeted perturbations and manipulations made to the input Audio data.Brief description of the accompanying drawings

[0008] An embodiment of the invention is described with reference to the following accompanying drawings:

[0009] Figure 1 depicts a processor to detect poisoning of an ASR model.

[0010] Figure 2 illustrates method steps to detect poisoning of an ASR model.Detailed description of the drawings

[0011] Automatic speech recognition (ASR) models process an audio into a readable text, herein referred to as a transcript. Typically, ASR models comprise of multiple components responsible for tasks like audio processing, feature extraction and language modelling. The state of the art comprises traditional ASR systems based on HMM (Hidden Markov Models) and GMM (Gaussian Mixture Models) that function on ‘force alignment’ that determines where in time particular words occur in the speech segment. The modem ASR technologies are transformer based (OpenAI whisper), LSTM based (DeepSpeech) and / or use supervised approach ( wav2vec 2.0) and the like.

[0012] The ASR models are trained on large-scale audio data. These ASR models are susceptible to poisoning attacks where an attacker deliberately manipulates the training data to degrade the performance of the ASR system which causes the ASR model to make incorrect predictions upon receiving certain inputs during inference. An example of such an attack is when an imperceptible noise or a structured noise is inserted into the input audio signal such that a specific target transcription is labelled according to the poisoned input signal. Consequently, during inference, if the same pattern is added to any future audio signal, the ASR model will output the specific target transcription. The present invention primarily focuses on detecting whether or not such an attack is directed to an ASR model. It is to be understood that the terms ‘clean-input’ and ‘test-input’ used hereafter refer to a large dataset or batches of data relating to audio speech. It is preferred that the same is interpreted with a context of a large dataset than as a single input signal.

[0013] Figure 1 depicts a system (10) to detect poisoning of a trained Audio speech recognition model (ASR model). The system (10) comprises the trained ASR model (2), a processor (1) and an autoencoder (3). The system is characterized by the functionality of the processor (1) and the autoencoder (3).

[0014] The components of the system (10) are further described in detail. In an exemplary embodiment of the present invention, the processor (1) may be implemented as any or a combination of: one or more microchips or integrated circuits interconnected using a parent board, hardwired logic, software stored by a memory device and executed by a microprocessor, firmware, an application specific integrated circuit (ASIC), and / or a field programmable gate array (FPGA). The processor (10) may comprise a storage such as RAM, On-chip cache, non-volatile-storage (SSD or HDD), SRAM / DRAM. According to exemplary embodiments, the processor (10) is configured to integrate Al models such as the ASR model and the Autoencoder by means of specialized Al units such as Neural Processing Unit, Tensor Processing Unit, Graphics processing Unit. In an alternate embodiment on the present invention the processor may reside remotely in a cloud.

[0015] The system further comprises the ASR model (2) and the autoencoder (3). According to an embodiment, the ASR model (2) and the autoencoder (3) are comprised in the storage of the processor. According to an embodiment, the ASR model (2), the processor(l) and the autoencoder(3) may communicate via local and / or remote processes such as in accordance with a signal having one or more data packets (e.g., data from one component interacting with another component in a local system, distributed system, and / or across a network such as the Internet with other systems via the signal).

[0016] The autoencoder (3), as known in the art, is a type of neural network often employed to learn efficient representations of data by compressing input data into a latent (compressed) space and then reconstructing it. According to the present disclosure, the autoencoder (3) is configured to reconstruct an input audio signal. The performance of the autoencoder is measured based on an input-loss-threshold (4). The input-loss-threshold (4) refers to a predefined limit on the acceptable level of loss between the original input and the output (reconstructed input). For the purposes of the present disclosure, according to an example, this input loss threshold determination includes a comparison of a signal to noise ratio (SNR) between the input fed to theautoencoder and the reconstructed input by the autoencoder (output of the autoencoder).

[0017] A person skilled in the art will understand that the system (10) as disclosed in the present disclosure is trained to detect poisoning before being deployed to infer poisoning of the ASR model (2). In order to train the system to detect poisoning of the ASR model (2), the processor (1) is configured to feed a clean-input (X) to an autoencoder (3) to generate a reconstructed clean-input (X Recons). The processor is further configured to feed the clean-input (X) to the ASR model (2) to obtain a transcript (Y). The processor is further configured to feed the reconstructed clean-input (X Recons) to the ASR model to obtain a reconstructed transcript (Y Recons).

[0018] The processor then compares the transcript (Y) with the reconstructed transcript (Y Recons) to determine a pre-defined threshold. Once the pre-defined threshold is determined, the processor detects poisoning of the ASR model based on the pre-defined threshold.

[0019] The pre-defined threshold is determined by various metrics of the output transcripts. These metrics, herein referred to as ‘output metrics’ comprise at least one of the following:• Word Error Rate (WER): It measures the percentage of words that were incorrectly recognized or missed.• Character Error Rate (CER): It measures the percentage of characters that were incorrectly recognized.• Sentence Error Rate (SER): The percentage of sentences that contain at least one error.• Word Accuracy (WA): Measures the percentage of correct words recognized.• Confidence score: Represents the ASR system's confidence in the accuracy of the recognized words or sentences.

[0020] According to the present disclosure, clean-input (X) refers to an input audio signal whose integrity is ascertained. That is, it is confirmed that the input audio signal is not poisoned. Once the pre-defined threshold is determined based on the output metrics, the processor detects poisoning of the ASR moder based on the pre-defined threshold.

[0021] In order to detect poisoning of the ASR model (2) based on the pre-defined threshold in situations when the integrity of the input data is not ascertained (herein referred to as ‘test-input’), the processor is configured to feed the autoencoder (3) with a test-input to obtain a regenerated test-input. The processor is further configured to feed the ASR model with the test-input to obtain a test-transcript. The processor is further configured to feed the regenerated test-input to the ASR model to obtain a reconstructed test-transcript and compare the output metrics of the test-transcript with the output metrics of the reconstructed test-transcript. The processor then detects poisoning of the ASR model (2) based on a deviation observed from the pre-defined threshold upon said comparison.

[0022] It should be understood at the outset that, although exemplary embodiments are illustrated in the figures and described below, the present disclosure should in no way be limited to the exemplary implementations and techniques illustrated in the drawings and described below.

[0023] The implementation of the present disclosure is described hereafter. Figure 2 illustrates method steps to detect poisoning of an ASR model. The method steps (100) are carried out in the system disclosed in accordance with figure 1. All components of the system (100) have been explained in accordance with figure 1.

[0024] Method step 101 comprises feeding an autoencoder with a clean-input to generate a reconstructed clean-input. The auto-encoder takes the clean-input, the clean-input being an audio signal whose integrity is ascertained. The autoencoder then reconstructs this clean-input. The autoencoder, as known in the art, is a type of neural network often employed to learn efficient representations of data by compressing input data into a latent (compressed) space and then reconstructing it. According to the present disclosure, the autoencoder is configured to reconstruct an input audio signal. The performance of the autoencoder is measured based on an input loss threshold. The input loss threshold refers to a predefined limit on the acceptable level of loss between the original input and the output (reconstructed input). For the purposes of the present disclosure, according to an example, this input loss threshold comprises a comparison of a signal to noise ratio (SNR) between the input fed to the autoencoder and the reconstructed input by the autoencoder (output of the autoencoder).

[0025] The method step (102) comprises feeding the ASR model with the clean-input to obtain a transcript. It is to be understood that the transcript refers to the text generated from the audio signal by the ASR model.

[0026] The method step (103) comprises feeding the ASR model with the reconstructed clean-input to obtain a reconstructed transcript. The reconstructed transcript from the reconstructed clean-input will not be similar to the transcript generated by the ASR model from the original clean-input. Comparison of these will provide a threshold range or an extent to which such a reconstruction by the autoencoder deviates the transcript. Therefore, the method step (104) comprises comparing the transcript with the reconstructed transcript to determine a pre-defined threshold. Once the pre-defined threshold is determined, step (105) comprises detecting poisoning of the ASR model based on the pre-defined threshold.

[0027] The step (104) comprises a further step (104a) of determining said pre-defined threshold based on a set of output metrics, said output metrics comprise at least one ofa Word Error Rate (WER), Character Error Rate (CER), Sentence Error Rate (SER), Word Accuracy (WA), and a Confidence Score

[0028] The step 105 of detecting poisoning of the ASR model based on the pre-defined threshold comprises further steps 1051, 1052, 1053, and 1054. The step (1051) is feeding the autoencoder with a test-input to obtain a reconstructed test-input. The test input refers to an unknown input audio signal whose integrity is not ascertained, that is, said test-input may be poisoned.

[0029] The step (1052) comprises feeding the ASR model with the test-input to obtain a test-transcript. The following step (1053) is feeding the reconstructed test-input to the ASR model to obtain a reconstructed test-transcript and step (1054) is comparing the output metrics of the test-transcript with the output metrics of the reconstructed test-transcript.

[0030] Step (1055) comprises detecting poisoning of the ASR model based on a deviation observed from the pre-defined threshold upon said comparison. According to an embodiment of the present disclosure, in addition to output metrics as disclosed above, an input metric of signal to noise ratio (SNR) is used. The signal to noise ratio is compared between the test-input and the reconstructed test input and can be used to determine the pre-defined threshold.

[0031] The working of the invention is further described. According to an example, the system is first trained on clean-input data. It is assumed that that the original input audio signal (clean input) is a speech -‘What’s an autoencoder’ and ASR model produces the transcript ‘What’s an autoencoder’ ‘from this clean-input. The same input audio signal is reconstructed by the autoencoder, wherein, during reconstruction, the signal to noise ratio of the audio signal changes. The ASR model generates a reconstructed transcript - ‘What is autoencoder’. The output metrics (such as WER, confidence score, word error rate, sentence error rate ) are measured and compared for the reconstructed transcript (from the reconstructed input) and the transcript (from the clean input) for a large set of audio data (clean input data). A pre-defined threshold isobtained based on these metrics. Now, assuming that an attacker inputs a poisonous input by introducing noise to it. In such a situation, the autoencoder, while reconstruction of this poisonous input will not be able to reconstruct data points that deviate from a learned distribution. This Teamed distribution’ is basically a compressed latent representation of the normal data distribution obtained from the large set of audio data used to train the autoencoder (clean input). This will lead to a high reconstruction error by the autoencoder. In other words, the reconstructed poisonous input will differ significantly from the original test data. Consequently, once passed through the ASR model, the output metrics of the reconstructed poisonous input and the original poisonous input will differ significantly allowing the output metrics to breach the pre-defined threshold, thereby detecting poisoning.

[0032] It must be understood that the invention discloses methodology used for poisoning of an ASR model. While these methodologies describe only a series of steps to accomplish the objectives, these methodologies are implemented in the system, which may be a combination of hardware or software or a combination thereof. Further the embodiments explained in the above detailed description are only illustrative and do not limit the scope of this invention. Any modification to the system and method for detecting poisoning of a ASR model form a part of this invention. The scope of this invention is limited only by the claims.

Claims

We Claim:

1. A method (100) to detect poisoning of an Audio speech recognition (ASR) model by a system, said system comprising:- the ASR model adapted to generate a transcript from an input audio signal, a processor, and an autoencoder, the method characterized by the following steps:- feeding an autoencoder with a clean-input to generate a reconstructed cleaninput (101),- feeding the ASR model with the clean-input to obtain the transcript (102),- feeding the ASR model with the reconstructed clean-input to obtain a reconstructed transcript (103),- comparing the transcript with the reconstructed transcript to determine a predefined threshold (104), and- detecting poisoning of the ASR model based on the pre-defined threshold (105).

2. The method (100) as claimed in claim 1, wherein, determining said predefined threshold (104) based on a set of output metric(104a), said output metrics comprise at least one of a :- Word Error Rate (WER),- Character Error Rate (CER)- Sentence Error Rate (SER),- Word Accuracy (WA),- Confidence Score3. The method(lOO) as claimed in Claim 1 wherein, detecting poisoning of the ASR model based on the pre-defined threshold (105) comprises the steps of :- feeding the autoencoder with a test-input to obtain a regenerated test-input (1051),- feeding the ASR model with the test-input to obtain a test-transcript (1052),- feeding the regenerated test-input to the ASR model to obtain a reconstructed test-transcript (1053),- comparing the output metrics of the test-transcript with the output metrics of the reconstructed test-transcript (1054), and-detecting poisoning of the ASR model based on a deviation observed from the pre-defined threshold upon said comparison (1055).

4. The method (100) as claimed in Claim 2, wherein, said pre-defined threshold is further determined based on an input metric comprising a signal to noise ratio (SNR) of the test input and the reconstructed test input.

5. A system (10) to detect poisoning of an Audio speech recognition (ASR) model, the system comprises:- the ASR model (2) adapted to generate a transcript from an input audio signal,- a processor (1), characterized in that system, the processor configured to:- feed a clean-input to an autoencoder (3) to generate a reconstructed clean-input,- feed the clean-input to an ASR model (2) to obtain a transcript,- feed the reconstructed clean-input to the ASR model (2) to obtain a reconstructed transcript,- compare the transcript with the reconstructed transcript to determine a pre-defined threshold, and- detect poisoning of the ASR model (2) based on the pre-defined threshold.

6. The system as claimed in Claim 1, wherein, to detect poisoning of the ASR model (2) based on the pre-defined threshold, the processor (1) is configured to :- feed the autoencoder (3) with a test-input to obtain a regenerated testinput,- feed the ASR model (2) with the test-input to obtain a test-transcript,- feed the regenerated test-input to the ASR model (2) to obtain a reconstructed test-transcript,- compare the output metrics of the test-transcript with the output metrics of the reconstructed test-transcript, and-detect poisoning of the ASR model (2) based on a deviation observed from the pre-defined threshold upon said comparison.Dated this 30th Day of September 2024 (Digitally signed)Siddharth Karkhanis (IN / PA- 1195)On behalf of the applicants.

Images