Automatic speech recognition systems that address perception-based adversarial audio attacks

Abstract

The present invention relates to an automatic speech recognition system that addresses perception-based adversarial audio attacks. A computer-implemented method for generating a combined audio signal in a speech recognition system, the method comprising the following steps: sampling an audio input signal to generate a time-domain sampled input signal, then converting the time-domain sampled input signal into a frequency-domain input signal, then generating perceptual weights in response to frequency components of critical bands of the frequency-domain input signal, generating a time-domain adversarial signal in response to the perceptual weights; and combining the time-domain adversarial signal with the audio input signal to generate a combined audio signal, wherein speech processing of the combined audio signal will output a result different from speech processing of the audio input signal.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Application No. 62 / 861,965, filed on June 14, 2019, the disclosure of which is incorporated herein by reference in its entirety. [Technical field]

[0003] The present invention generally relates to an automatic speech recognition system (ASR) that addresses perception-based adversarial audio attacks by adding relatively small, imperceptible amounts of noise to the signal. [Background technology]

[0004] Automatic speech recognition (ASR) is a technology that recognizes and translates spoken language into text by a computer system. Spoken language or speech is received via a microphone or other input and processed to generate text associated with the speech. The text can be used as input to a computer, computer system, or other system via the interface so that the system can be controlled using normal human conversation.

[0005] Adversarial machine learning is a technique used in the field of machine learning that attempts to trick an ASR model with malicious inputs. Adversarial machine learning can be applied to attack machine learning models or cause failures in machine learning models. However, adversarial machine learning can also have beneficial applications, such as improving the robustness of machine learning models. These techniques can be used to manipulate input data in order to identify vulnerabilities in learning algorithms, such as security measures of machine learning systems. Once these security vulnerabilities are known, they can be used to improve the robustness of machine learning systems. [Summary of the invention]

[0006] A computer-implemented method for obtaining an adversarial signal associated with an audio input signal to a speech recognition system, the method comprising: sampling the audio input signal to generate a time-domain sampled input signal, then converting the time-domain sampled input signal into a frequency-domain input signal, then generating perceptual weights based on frequency components of critical frequency bands of the frequency-domain input signal, generating a time-domain adversarial signal based on the perceptual weights; and combining the time-domain adversarial signal with the audio input signal to generate a combined audio signal, wherein speech processing of the combined audio signal will output a result different from speech processing of the audio input signal.

[0007] A non-transitory computer-readable medium tangibly embodying computer-readable instructions for a software program executable by a processor of a computing device to provide operations including: sampling an audio input signal to generate a time-domain sampled input signal, converting the time-domain sampled input signal to a frequency-domain input signal, generating perceptual weights based on frequency components of critical bands of the frequency-domain input signal, generating a time-domain adversarial signal based on the perceptual weights, and combining the time-domain adversarial signal with the audio input signal to generate a combined audio signal.

[0008] A computer system for performing an adversarial audio attack on an automated speech recognition system, the computer system having a processor for executing computer-readable instructions and a memory for maintaining computer-executable instructions, which when executed by the processor perform the following functions: sampling an audio input signal to generate a time-domain sampled input signal, converting the time-domain sampled input signal to a frequency-domain input signal, identifying critical bands via a psychoacoustic model of the human ear, generating perceptual weights based on frequency components of the critical bands of the frequency-domain input signal, generating a time-domain adversarial signal based on the perceptual weights, and combining the time-domain adversarial signal with the audio input signal to generate a combined audio signal.

Brief Description of the Drawings

[0009] Figure 1 Depicted is a five (5) step flow chart for calculating a global masking threshold according to one embodiment.

[0010] Figure 2 is a graphical representation of the absolute hearing threshold relative to the global masking threshold extracted from the speech frame.

[0011] Figure 3 is a graphical representation of the Perceptual Evaluation of Speech Quality (PESQ) scores associated with varying α.

[0012] Figure 4A is the spectrogram of the original audio sample.

[0013] Figure 4B is the spectrogram of the original audio sample with α=1.

[0014] Figure 4C is the spectrogram of the original audio sample with α=0.5.

[0015] Figure 5A is a graph of the word error rate (WER) using the language model (LM) decoder for air attack.

[0016] Figure 5B is a graph of the character error rate (CER) using the LM decoder for an over-the-air attack.

[0017] Figure 5C is a graphic representation of WER using the greedy decoder for an aerial attack.

[0018] Figure 5D is an illustration of CER using a greedy decoder for an over-the-air attack.

[0019] Figure 6 is a diagram of an audio adversarial attack environment illustrating changes in speech recognition output.

[0020] Figure 7 is a schematic diagram of a computing platform that can be used to implement the ASR formulas and algorithms of one or more embodiments. [Specific implementation method]

[0021] As required, detailed embodiments of the present invention are disclosed herein, however, it should be understood that the disclosed embodiments are merely examples of the present invention that may be specifically implemented in various alternative forms. The drawings are not necessarily to scale; some features may be exaggerated or minimized to show details of specific components. Therefore, the specific structural and functional details disclosed herein are not to be interpreted as limiting, but are merely to be interpreted as a representative basis for teaching those skilled in the art to adopt the present invention differently.

[0022] The term "substantially" may be used herein to describe disclosed or claimed embodiments. The term "substantially" may modify a value or relative property disclosed or claimed in the present disclosure. In this case, "substantially" may mean that the value or relative property it modifies is within ±0%, 0.1%, 0.5%, 1%, 2%, 3%, 4%, 5%, or 10% of the value or relative property.

[0023] Adversarial audio attacks alter the output of an automatic speech recognition system (ASR) by adding small, imperceptible amounts of noise to the signal. These attacks can be carefully crafted using psychoacoustic properties.

[0024] Previous methods suffer from instabilities during back-propagation in the frequency domain. These methods are not explored in the physical domain, i.e., they are performed in a purely digital domain. The success of the adversarial can be severely affected in aerial scenarios due to path effects from the speaker to the receiver as well as additive noise.

[0025] A psychoacoustic based attack in the frequency domain that does not have the aforementioned stability issues is disclosed. The psychoacoustic based attack exploits the symmetric properties of the Discrete Fourier Transform (DFT) so that the attack can be performed only in the time domain once the relevant perceptual metrics are extracted from the original audio. This reduces the storage requirements of the attack compared to the frequency domain attack since both the real and imaginary components do not need to be stored. Further, the addition of a generated room impulse response yields robustness to the attack against air conditions.

[0026] While machine learning algorithms have demonstrated impressive performance on a variety of tasks in recent years, it has been shown that these algorithms may be subject to so-called adversarial attacks. These adversarial attacks have been applied to a variety of tasks including image, speech, and control domains. In an adversarial attack, small perturbations (usually imperceptible to human observers) are added to the input signal with the goal of changing the output of the algorithm. This has raised concerns not only about edge cases that may lead to unexplained failures, but also about the safety of systems that employ these algorithms.

[0027] With the ubiquity of personal voice assistants (e.g., Siri, Alexa, Google Home, etc.) that rely on automatic speech recognition (ASR) systems, it is a natural question to ask how robust these systems are to adversarial attacks. Typical adversarial attacks on ASR systems are generated through targeted attacks that change characters in the original utterance so that a target phrase is transcribed. While there are cases where a single character or word replacement may be sufficient, for most adversarial attacks on ASR systems, entire phrases need to be attacked. Furthermore, adversarial attacks must be robust to perturbations such that they can be played over the air.

[0028] In the audio domain, the psychoacoustic properties of the human auditory system can be exploited to generate more powerful but less perceptible attacks. These rely on masking effects, where certain frequencies are less perceptible when presented to the auditory system simultaneously. Another benefit of relying on perception-based metrics is that any non-speech audio in the environment can be used to inject noise into the signal, i.e., more noise allows for stronger attacks. For example, this technique has been used for attacks based on musical compositions containing adversarial voice commands.

[0029] Aerial attacks have been successfully implemented by incorporating transformations that audio signals typically encounter from source to sink. However, this adversarial generation does not incorporate any psychoacoustic properties that could produce audible distortions to the signal. These psychoacoustic properties rely on backpropagation through the frequency domain and are not performed in an aerial scenario. Furthermore, most previous adversarial audio scenarios rely on subjective human listening tests and measurements that do not explicitly consider the perceptual distortion of the adversarial signal, for example, using only signal-to-noise ratio (SNR) differences.

[0030] This disclosure provides a new formulation for psychoacoustic-based attacks on adversarial audio generation. The formulation exploits the psychoacoustic masking threshold of the original signal and has a clear tradeoff between audio distortion and the strength of the adversary. The adversary can be generated entirely in the time domain (once the psychoacoustic properties of the original phrase are converted from the frequency domain). The adversarial audio can be evaluated using Perceptual Evaluation of Speech Quality (PESQ) scores, which can eliminate the need for human listening tests. Finally, over-the-air tests are performed to highlight the robustness of the adversary.

[0031] A typical deep learning based ASR system is an end-to-end model that takes a raw speech clip and outputs a transcript. Since the processing of audio signals is a multi-step process, the attack can be directed at the original signal or at the signal after the original signal has been processed in the transform domain, such as after a Fast Fourier Transform (FFT) operation. To further increase the effectiveness of the attack, the attack can be applied in such a way that the characteristics of the signal are exploited in both the time and frequency domains.

[0032] The attacks are generated by manipulating the Mel frequency cepstral coefficients (MFCC) and inverse transforming the coefficients back to the time domain. However, due to the lossy nature of MFCC calculation, some information of the original audio signal may be lost, which limits the attack space. The original signal is attacked in a completely end-to-end manner. These attacks focus on limiting the size of the adversary based solely on the signal-to-noise ratio (SNR), without considering the perceptual distortion introduced by the adversary.

[0033] Further, in one or more embodiments, the attack is extended to incorporate typical transformations encountered by the audio signal from source to receiver during adversarial generation. This extension can improve the robustness of airborne broadcast attacks, but is more perceptible to listeners. Psychoacoustic properties can be used to reduce the perceptibility of adversarial signals. However, both methods may encounter instabilities during back propagation in the frequency domain. These methods have not been explored in the physical domain, that is, they have been explored in the purely digital domain. The success of the adversary may be severely affected in airborne scenarios due to path effects from the speaker to the receiver and additive noise.

[0034] Here, a formulation for a psychoacoustic based attack is proposed that addresses one or more of the above stability issues in the frequency domain. The formulation exploits the symmetric properties of the Discrete Fourier Transform (DFT) so that the attack can be performed solely in the time domain once the relevant perceptual metrics are extracted from the original audio. This reduces the storage requirements of the attack compared to the frequency domain formulation, since both the real and imaginary components do not need to be stored. By using the generated room impulse response, we further increase the robustness of the attack to aerial conditions.

[0035] For perceptual evaluation of the adversarial signal, the Perceptual Evaluation of Speech Quality (PESQ) score is used (with a range from 0.5 (poor) to 4.5 (excellent). The PESQ score is an objective measure for evaluating the speech quality of a system that causes distortion of a reference speech signal in a fully automatic manner, i.e., without the need for a listener. It therefore helps characterize the quality of a speech signal as it would be perceived by a listener.

[0036] The generated attacks are shown to have high perceptual scores, i.e., low audible distortion, while still changing the ASR system's transcription. The adversarial attacks are also shown to be robust when played through speakers in both mono (original audio and adversarial on the same channel) and stereo (original audio and adversarial on different channels).

[0037] Model Architecture

[0038] In one or more embodiments, the DeepSpeech model is used to generate adversarial audio examples. DeepSpeech is a speech-to-text multi-layer bidirectional model that uses connectionist temporal classification (CTC) loss. The DeepSpeech model is fed with a raw audio waveform x to produce character-level probabilities for each frame, which when decoded produces an output phrase y. The CTC loss function is also able to score the most likely characters and phrases over different temporal alignments of x. CTC is a fully differentiable function that can therefore be exploited to allow adversarial generation at each character level over each temporal instance over the entire length of x.

[0039] Adversarial Audio Generation

[0040] In adversarial audio attacks, a perturbation δ is applied to the original waveform, So that the output now becomes the target phrase y t This can be formulated as an optimization problem on δ in the form illustrated in equation (1) below:

[0041]

[0042] where L(·) is the loss function and where |·|2 is the l2 norm. This minimization problem is solved over the entire audio signal again by utilizing the CTC loss function, which ensures that the adversarial l2 norm is inside some ε-sphere.

[0043] The adversary in equation (1) is restricted only to the interior of the ε-sphere and is usually chosen in such a way as to minimally perturb the input. However, even with a successful attack using this formula, audible artifacts may still be perceptible. This is a result of adding certain frequency components when considering the attack from the frequency domain, where the human auditory system has a sensitivity to changes in intensity as a function of frequency. This sensitivity can be included in the optimization of equation (1) and exploited to better mask the adversary.

[0044] Psychoacoustic Model

[0045] The sensitivity of the human auditory system is a function of both intensity, which is usually measured in logarithmic sound pressure level (dBSPL), and frequency. It does not have a uniform response, which requires as little as -5dB SPL (light leaf rust) at peak conditions (2kHz to 5kHz), and the human auditory system requires higher intensities, especially as the boundaries of human hearing ([20Hz, 20kHz]) are approached.

[0046] Due to this sensitivity and the discrete way the auditory system processes sound, a masking effect occurs around critical bands when multiple frequencies are presented to the listener simultaneously. Critical bands can be thought of as drowning out other frequencies in the neighborhood, which is also a function of frequency and intensity, i.e., low intensities produce less masking effects. This masking can therefore be exploited to embed a countermeasure signal below a certain hearing threshold, ensuring that it remains imperceptible.

[0047] Speech can be considered as a process that changes dynamically over the entire time domain. In order to obtain an accurate representation of the frequency components, short segments or frames are usually analyzed. For speech processing, the frame length is usually on the order of 10ms to 20ms, but can be larger or smaller, where the frequency components are assumed to be fixed within the time frame. There is a small amount of overlap between frames to ensure frequency continuity, and a window function is applied to smooth the transition of frequency components. Using this method, the original waveform x is divided into N frames of length L, as given by the following equation (2):

[0048] x n (kT) = x(kT + nL)w L (t-nL) k∈[0,N-1] (2)

[0049] where n is the frame index, and w L is the window function.

[0050] The psychoacoustic model used to find the global masking threshold is based on MPEG-ISO and is included in the attack. In one or more embodiments, the global masking threshold is generated via a five (5) step calculation. Calculating the global masking threshold for each frame includes the following five (5) steps. Figure 1 A flowchart 10 of five (5) steps for calculating a global masking threshold is depicted according to one embodiment.

[0051] Step 1

[0052] According to block 12 of flow chart 10, the frames are first normalized to a standard dB SPL, which converts each frame to approximately the same intensity level. Although this is only an approximation of dB SPL, it is required because the signal has intensity levels that are a function of room dynamics, microphone response, etc. The signal is then windowed and transformed to the FFT domain to generate a power spectral density (PSD).

[0053] Step 2

[0054] According to block 14 of flow chart 10, tonal and non-tonal masking signals are then identified in the PSD. Tonal masking signals represent precise frequency peaks in the PSD, while non-tonal masking signals are found by taking the geometric mean across a set of frequencies. These masking signals then help identify which frequencies become less perceptible when presented simultaneously.

[0055] Step 3

[0056] According to block 16 of flowchart 10, the number of masked signals is then reduced or decimated by comparing the tonal masked signals and non-tonal masked signals using a sliding window scheme. This reduces the granularity of the masked signal and results in smoother transitions between peaks in the PSD.

[0057] Step 4

[0058] According to block 18 of flow chart 10, the tone masker signal and the non-tone masker signal are then used to generate a masking pattern covering adjacent frequencies.

[0059] Step 5

[0060] According to block 20 of flowchart 10, a global masking threshold is then determined by combining the masking patterns from the previous steps. The global masking threshold then represents a perceptual weighting based on the intensity and frequency components of the signal and the psychoacoustic properties of the human auditory system.

[0061] The resulting global masking threshold t can then be found for each frame N across all frequencies f, Among them, f s is the sampling frequency.

[0062] Figure 2The absolute threshold of hearing (minimum dB SPL for the sound to be perceived), tonal and non-tonal masking signals, and the global masking threshold for a single analysis frame are shown. The fill between the lines represents the portion of the frequency band that will not be perceived unless a higher dB SPL is reached, for example, around 3kHz there is a gap of 10 dB SPL between the global masking threshold and the PSD of the analysis frame. Therefore, the adversarial signal can be added to this portion of the spectrum until the masking threshold is reached while remaining inaudible.

[0063] Perception-based audio attack optimization

[0064] Since psychoacoustic models exploit the frequency domain relationship between signals and the human auditory system, it is natural to first analyze how the attack can be formalized in the frequency domain. Relying on the linear properties of the Fourier transform, the adversary at frame n and frequency f can be expressed as set forth in equation (3) below.

[0065]

[0066] The perceptual loss can be formulated as a weighting in the frequency domain between the original waveform and the adversarial waveform represented as set forth in equation (4) below.

[0067]

[0068] Among them, w n (f) is a perceptually based weighting factor. This weighting factor is based on a global masking threshold derived in a psychoacoustic model as set forth in equation (5) below.

[0069] w n (f) = 10-βt n (f) (5)

[0070] Where β is a compression factor that defines how much perceptual-based weighting should be included, and how much perceptual-based weighting has been heuristically determined to be in the range [0.04-0.06]. This can be thought of as analogous to the power weighting between the original signal and the adversary.

[0071] The frequency-based perceptual loss can now be reformulated in the time domain as set forth in equation (6) below:

[0072]

[0073] where (·)H represents the conjugate transpose of the matrix, D is the DFT matrix, and W n is defined as follows in equation (7) around The symmetric weighting matrix of :

[0074]

[0075] For ease of explanation, the multiplication of the weighting matrix and the DFT matrix is ​​represented as G n =D H W n D. Note that due to the symmetric nature of the DFT, the imaginary components cancel during multiplication, making G n ∈R L×L , where L is the DFT window length, which is assumed to be equal to the frame length in equation (2).

[0076] Now we can combine the perceptual-based loss in Equation (6) and the adversarial loss in Equation (1) as shown in Equation (8):

[0077]

[0078] Among them, 0≤α≤1, and L tot Denotes the total loss. While ε constrains the total size of the perturbations, α controls the amount of perceptual-based penalty included in the optimization.

[0079] One benefit of using equation (8) is that since the original audio is not altered, i.e., the global masking threshold remains constant throughout the optimization process, the G matrix only needs to be calculated once before optimizing the adversary. This means that the adversarial signal does not need to be transformed to the frequency domain at each iteration. Compared to previous attacks that occurred in the frequency domain, this method reduces the required storage because the real and complex components are no longer stored, and alleviates the stability issues in the optimization mentioned earlier.

[0080] Using the l2 norm constraint on δ in Equation (8) allows the attack to be more localized in the audio signal. This is in contrast to using l ∞ In contrast to other adversarial audio attacks, ∞ allows equally large perturbations across the entire audio signal. Since the adversarial is now constrained based on the l2 norm, the adversarial is generally more robust than l ∞ Attacking larger values. Equation (8) can be solved using the projected gradient descent (PGD) method as described below in equation (9).

[0081]

[0082] Among them, P ε is the projection onto the ε sphere, and μ is the step size. The projection of equation (9) can then be expressed as set forth in equation (10) below.

[0083]

[0084] Before projection, we also include the constraint -1 ≤ x + δ ≤ 1 to ensure that the adversarial signal lies within the valid normalized audio signal range. The normalization of the gradient in Eq. (9) also helps stabilize the descent direction.

[0085] Experimental Results

[0086] The DeepSpeech model is trained in pyTorch using the Librispeech dataset, which consists of 960 hours of clean audio and corresponding transcriptions. During training, the f s =16kHz sampling frequency, and the data is augmented by adding random noise and pitch and time-shifting the signal. For all experiments, the compression factor in equation (5) is β=0.06. The probabilities from the DeepSpeech model are decoded using a greedy approach, i.e., in each case, only the character with the highest probability is selected.

[0087] To evaluate the performance of the attack, we use several measures that analyze both the signal characteristics and the final transcription. The word error rate (WER) is derived from the Levenshtein distance algorithm and defines the minimum edit between two strings given by the following equation (11):

[0088]

[0089] where S is the number of substitutions, D is the number of deletions, I is the number of insertions, and N = S + D + C, where C is the number of correct words. For a perfect attack, y = y t , S = D = I = 0, resulting in WER = 0. As the distance between the two strings increases, y≠y t , i.e., more characters and words are changed, the WER also increases. The character error rate (CER) is the per-character difference between two strings, and when y=y t When CER=0. For perceptual evaluation, we use the Perceptual Evaluation of Speech Quality (PESQ) score with a range from 0.5 (poor) to 4.5 (excellent). The PESQ score is calculated in full reference mode and is the time-aligned x and The output SNR is estimated using the original signal and δ as the noise signal.

[0090] Use the phrase "open the door" as a tThe generated adversarial attack with ε=1000 has a 100% success rate (WER=CER=0) on 100 randomly sampled audio files from the Librispeech test set when no perceptual weighting (α=1) is used. The same randomly sampled files are attacked again, this time (α=0.8) allowing perceptual weighting to be included in the loss function. The perceptually weighted files again have a 100% success rate (WER=CER=0).

[0091] SNR and PESQ scores are calculated for each signal using a perceptual-based attack (α=0.8) and a non-perceptual-based attack (α=1). In Table 1, it is shown that the SNR decreases while the PESQ score increases when a perceptual-based attack is used compared to a non-perceptual-based attack. When perceptual weighting is not used, the attack can be equally spread anywhere on the spectrum. This spreading in the frequency domain reduces the total amplitude of any one frequency component corresponding to a lower adversarial signal power and therefore a higher SNR. However, when perceptual loss is included, the attack is more concentrated around the frequencies that dominate the global masking threshold. This has the effect of increasing the power on some components, reducing the SNR but ensuring that these components are within the masking threshold and are inaudible, which improves the PESQ score.

[0092]

[0093] Figure 3 The PESQ score is illustrated as α decreases, the decrease stops when CER≠0 at α=0.3. Surprisingly, the PESQ score rises rapidly with only a small decrease in α, indicating a rapid increase in perceived quality. With α=0.4, the PESQ score is almost at the maximum value of 4.5, so to the listener it sounds like We observe that the attack does not always succeed for very low values ​​of α. This is most likely due to the adversary being heavily penalized for being outside the global masking threshold. This limits the total bandwidth the attack can use and can prevent it from changing the signal enough to generate a successful attack.

[0094] The effect of α can also be seen in FIG. 4A to FIG. 4C The changes in the spectrogram are observed. Figure 4B The spectrum plot is shown when α=1, so there is no constraint on where the adversary can attack. We see that the adversary is almost evenly spread around all frequency bands. Figure 4C When α = 0.5, we see that most of the attacks are located at these higher frequencies, especially when there is no direct source of Figure 2 The absolute hearing thresholds for speech are shown.

[0095] Air Attack

[0096] In one embodiment, the test is extended beyond the purely digital domain and an air attack (speaker / microphone combination) is performed. To increase the robustness of the signal to air attacks, the signal is first passed through a room simulator, which is very similar to the transform-expected method. Note that instead of using a pre-recorded room impulse response as in this method, a more robust approach is taken as in this method, where a room simulator is used to generate a response with varying magnitude and surface reflection coefficients. A bandpass filter is also applied in the range of f = [100, 7500] to eliminate any parasitic low frequency components and to account for the frequency drop that occurs around the Nyquist rate in digital recording hardware.

[0097] Loudness was measured using an Extech HD6000 sound level meter that uses A-weighted decibels (dBA) from the speaker to the microphone that takes into account human hearing sensitivity. Over-the-air testing was performed in an anechoic chamber with a noise floor of 36 dBA. The signal was broadcast using Polk S20 speakers, Yamaha p2500s amplifiers, and recorded using a MINI DSP UMIK-1 microphone.

[0098] The microphone is first positioned close to the speaker to ensure a high SNR and eliminate any path effects between the microphone and the speaker. Then, the distance between the microphone and the speaker is gradually increased, which results in a lower SNR, to observe the effect of increased path effects and reduced signal power. A language model decoder based on Librispeech 4-gram ARPA is also added to the end of the DeepSpeech model in parallel with the greedy decoder. The language model has the ability to perform error correction on characters and words based on a speech corpus, and in some cases, can reduce WER and CER rates.

[0099] FIG. 5A to FIG. 5D The WER and CER of the over-the-air attack using both the greedy and language model decoders are illustrated. At high SNR, the signal experiences clipping (the maximum amplitude of the signal is larger than the microphone response), which results in high WER and CER (y vs. y t Since the adversary is given as x+δ, it is shown that when clipping occurs on the positive track max{1,x+δ}, this will result in truncation of the adversary and, in extreme cases, of the original signal. The same will also apply to the negative track. As the distance increases and the SNR decreases (which moves x+δ off track), the signal exhibits lower WER and CER for the language model (LM) and greedy decoder (G). Between 60dBA and 70dBA, the language model decoder has WER=4.0 and CER=2.4, while the greedy decoder has WER=3.0 and CER=2.25.

[0100]

[0101] In one embodiment, the over-the-air scenario is extended to a two speaker setup broadcasting the original audio and the adversarial on two separate channels. The speakers are separated by a distance of 4 inches and the microphone is placed 6 inches from the speakers, which results in a separation of 40° between the speakers as observed from the microphone. The signal is measured at the microphone during the broadcast with an average of 66.7 dBA. Table 2 shows the average error rate for 4 trials for the language model and greedy decoder. Using the language model decoder for the two speaker system, the attack performs particularly well, achieving WER=0 and CER=0 for 3 out of 4 trials.

[0102] In one or more embodiments, a method for generating white-box adversarial examples with psychoacoustic-based constraints is disclosed. The adversarial examples are generated in an end-to-end manner, which relies on extracting relevant psychoacoustic information from the original audio signal. The optimization problem relies on l2-norm constraints combined with projected gradient descent. The perceptual quality is based on PESQ scores, which eliminates the need for exhaustive listening tests. The adversarial examples typically degrade the SNR, but due to the psychoacoustic-based constraints, the perceptual quality of the signal improves, which leads to higher PESQ scores. Finally, the attack is shown to be effective in over-the-air broadcasts.

[0103] Figure 6 is a diagram of an audio adversarial attack environment 50 including a speaker 52 and an adversary 54 . Figure 6 An original audio message "Turn off the TV" 56 from the speaker 52 and an audio message 58 from the adversary 54 are illustrated. The original audio message 56 and the audio message 58 are combined to generate a combined signal 60, thereby changing a first output of an automatic speech recognition (ASR) engine 62 between the original audio message "Turn off the TV" 56 and a second output 64 of the ASR engine 62 of the combined signal 60 with the adversary audio attack "Open the front door" 64.

[0104] The use of perceptually based adversarial audio attacks has applications in multiple fields, including automation and security of health care, vehicle systems, buildings and homes, and other voice-activated services. Perceptually based adversarial audio attacks can be used to improve the robustness of automated speech recognition systems by injecting signals that are imperceptible to ordinary people, but if others try to circumvent or otherwise attack the system, the additional signal can indicate an error. For example, in a hospital or health care environment, a voice-activated system can be used to generate prescriptions. The use of this perceptually based adversarial audio attacks can be used to supplement the speaker's natural voice, thereby increasing robustness. This also increases the security of preventing other adversarial attacks on the system. This can be implemented on a system separate from an automated speech recognition system, for example, a computer can have an automated speech recognition system, and can be a separate computer, smart phone or other computerized system for this purpose.

[0105] The program code that specifically implements the formulas, algorithms and / or methods described herein can be distributed separately or collectively as a program product in various forms. The program code can be distributed using a computer-readable storage medium having computer-readable program instructions thereon, which are used to cause a processor (such as Figure 7 The processor 102 of the computer readable storage medium performs one or more aspects of the embodiments. Inherently non-transitory computer-readable storage media may include volatile and nonvolatile, and removable and non-removable tangible media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media may also include RAM, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid-state storage technology, portable compact disk read-only memory (CD-ROM) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be read by a computer. Computer-readable program instructions can be downloaded from a computer-readable storage medium to a computer-readable storage medium such as a computer-readable storage medium. Figure 7 Computer readable storage medium 104), a computer (such as Figure 7 computer 100), another type of programmable data processing device, or another apparatus, or downloaded to an external computer or external storage device via a network.

[0106] The computer-readable program instructions stored in the computer-readable medium can be used to instruct a computer, other types of programmable data processing equipment or other devices to function in a particular manner so that the instructions stored in the computer-readable medium produce an article of manufacture including instructions for implementing the functions, actions and / or operations specified in the flow chart or diagram. In some alternative embodiments, the functions, actions and / or operations specified in the flow chart and diagram can be reordered, serially processed and / or concurrently processed in accordance with one or more embodiments. Moreover, any one of the flow chart and / or diagram may include more or fewer nodes or boxes than those illustrated according to one or more embodiments.

[0107] Although the present invention has been illustrated by the description of various embodiments, and although these embodiments have been described in considerable detail, the applicant does not intend that the scope of the appended claims will be limited to such details or in any way limited to such details. Additional advantages and modifications will be readily presented to those skilled in the art. Therefore, the present invention in its broader aspects is not limited to the specific details, representative devices and methods, and illustrative examples shown and described. Therefore, it is possible to deviate from such details without departing from the spirit or scope of the general inventive concept.

Claims

1. A computer-implemented method for obtaining an adversarial signal associated with an audio input signal to a speech recognition system, the method comprising: Sampling the audio input signal to generate a time-domain sampled input signal; Converting the time-domain sampled input signal into a frequency-domain input signal; generating perceptual weights according to frequency components of critical bands of the frequency domain input signal; generating a time domain adversarial signal according to the perception weight; as well as combining the time domain adversarial signal with the audio input signal to produce a combined audio signal, the time domain adversarial signal being a combination of a perceptual loss and an adversarial loss; inputting the combined audio signal into the speech recognition system; and The speech recognition system outputs a second output different from the audio input signal.

2. The computer-implemented method of claim 1 , wherein: The combination of the perceptual loss and the adversarial loss is represented by the following equation: Where 0≤α≤1, x is the original audio waveform, δ is the perturbation, and y target is the target phrase, L(·) is the loss function, L percep denotes the perceptual loss, Ltot denotes the total loss, ε constrains the total size of the perturbations, and α controls the amount of perceptual-based penalty included in the optimization.

3. The computer-implemented method of claim 2, wherein: An L2 norm constraint is used on the perturbation δ, and the combination of the perceptual loss and the adversarial loss is solved via a projected gradient descent (PGD) method.

4. The computer-implemented method of claim 1 , wherein: The time-domain sampled input signal is converted into the frequency-domain input signal via a symmetric algorithm, which includes a fast Fourier transform, a discrete Fourier transform or a symmetric filter bank.

5. The computer-implemented method of claim 1 , wherein: The audio input signal is sampled via an analog-to-digital A / D converter.

6. The computer-implemented method of claim 1 , further comprising the steps of: The critical frequency bands are identified via a psychoacoustic model of the human ear.

7. The computer-implemented method of claim 6, wherein: The psychoacoustic model is an MPEG psychoacoustic model or an AAC psychoacoustic model.

8. A non-transitory computer readable medium storing computer readable instructions for causing the computer to perform a method for obtaining an adversarial signal associated with an audio input signal to a speech recognition system, the method comprising: Sampling the audio input signal to generate a time-domain sampled input signal; Converting the time-domain sampled input signal into a frequency-domain input signal; generating perceptual weights according to frequency components of critical bands of the frequency domain input signal; generating a time domain adversarial signal according to the perception weight; as well as The time-domain adversarial signal is injected into the speech recognition system to produce different results from the speech recognition system, the time-domain adversarial signal is a combination of the perceptual loss and the adversarial loss, and the different results indicate an error of the speech recognition system.

9. The non-transitory computer readable medium of claim 8, wherein: The computer readable instructions are executable by a processor of the computing device to provide further operations including identifying the critical frequency bands via a psychoacoustic model of the human ear.

10. The non-transitory computer readable medium of claim 8, wherein: The combination of the perceptual loss and the adversarial loss is represented by the following equation: Where 0≤α≤1, x is the original audio waveform, δ is the perturbation, y target is the target phrase, L(·) is the loss function, L percep denotes the perceptual loss, Ltot denotes the total loss, ε constrains the total size of the perturbations, and α controls the amount of perceptual-based penalty included in the optimization.

11. The non-transitory computer readable medium of claim 10, wherein: The audio input signal is sampled via an analog-to-digital A / D converter.

12. A computer system for obtaining an adversarial signal associated with an audio input signal to a speech recognition system and for injecting the adversarial signal into the speech recognition system, the computer system having a processor configured to execute computer readable instructions and a memory configured to execute the computer executable instructions, the computer executable instructions when executed by the processor performing the following functions: Sampling the audio input signal to generate a time-domain sampled input signal; Converting the time-domain sampled input signal into a frequency-domain input signal; Identifying critical frequency bands via a psychoacoustic model of the human ear; generating perceptual weights based on frequency components of the critical bands of the frequency-domain input signal; generating a time-domain adversarial signal based on the perception weights; as well as The time domain adversarial signal is injected into the speech recognition system to produce a different result from the speech recognition system than a result produced when the audio input signal is injected into the speech recognition system, the time domain adversarial signal being a combination of a perceptual loss and an adversarial loss, and the different result indicating an error of the speech recognition system.

13. The computer system according to claim 12, wherein: The combination of the perceptual loss signal and the adversarial loss is represented by the following equation: Where 0≤α≤1, x is the original audio waveform, δ is the perturbation, y target is the target phrase, L(·) is the loss function, L percep denotes the perceptual loss, Ltot denotes the total loss, ε constrains the total size of the perturbations, and α controls the amount of perceptual-based penalty included in the optimization.

14. The computer system of claim 13, wherein: The audio input signal is sampled via an analog-to-digital A / D converter.

15. The computer system according to claim 12, wherein: The computer executable instructions, when executed by the processor, perform the following additional functions: identifying the critical frequency bands via a psychoacoustic model of the human ear.

16. The computer system of claim 15, wherein: The psychoacoustic model is an MPEG psychoacoustic model or an AAC psychoacoustic model.

17. The non-transitory computer readable medium of claim 9, wherein: The psychoacoustic model is an MPEG psychoacoustic model or an AAC psychoacoustic model.

18. The non-transitory computer readable medium of claim 10, wherein: An L2 norm constraint is used on the perturbation δ, and the combination of the perceptual loss and the adversarial loss is solved via a projected gradient descent (PGD) method.

19. The computer system of claim 13, wherein: An L2 norm constraint is used on the perturbation δ, and the combination of the perceptual loss and the adversarial loss is solved via a projected gradient descent (PGD) method.

Images