A speech naturalness assessment method for speech adversarial attacks

Abstract

The present invention discloses a method for evaluating the naturalness of speech against speech adversarial attacks, and belongs to the field of speech recognition model security technology in intelligent speech systems. Common features of adversarial audio are obtained and common features related to unnaturalness are screened, including discontinuous time domain signals, discontinuous frequency domain signals, irregular time domain signals, irregular frequency domain signals, and abnormal speech patterns; for each common feature, the acoustic-statistical features related to unnaturalness of benign speech samples and test speech are calculated; a speech naturalness index is designed according to the acoustic-statistical features related to unnaturalness of benign speech samples, and the reference value of the speech naturalness index is the probability density distribution of the acoustic-statistical features related to unnaturalness of benign speech samples; according to the reference value of the speech naturalness index, the naturalness score of the test speech is calculated. The present invention can quantify the similarity between the test speech and normal human speech by evaluating the naturalness of the speech sample.

Description

Technical Field

[0001] The present invention belongs to the field of voice recognition model security technology in intelligent voice systems, and in particular relates to a voice naturalness assessment method for voice adversarial attacks. Background Art

[0002] Speech recognition technology can convert speech into text. It has been widely used in devices such as smartphones, cars, and industrial control terminals, and can realize functions such as voice control, voice transcription, and voice search. Existing research shows that speech recognition technology is threatened by adversarial attacks. In adversarial attacks, attackers superimpose maliciously constructed adversarial noise on human speech to form adversarial samples, inducing the speech recognition model to output incorrect text. In the voice control scenario, such attacks can cause malicious operations on devices such as smartphones, cameras, home appliances, and vehicles, threatening the safe operation of various speech recognition applications.

[0003] Existing research generally generates adversarial samples by solving optimization problems. In this optimization problem, the optimization goal is that the audio plus the perturbation will be recognized by the model as the target text and the perturbation has low audibility to humans. When expressed in mathematical formulas, the optimization goal is to minimize the weighted sum of the distance between the model recognition result of the adversarial sample and the target text and the audibility index of the perturbation. In order to improve the concealment of the attack and increase the naturalness of the adversarial samples, existing research has optimized the audibility index of the perturbation, and has proposed a variety of schemes such as decibel distortion, absolute hearing threshold, and psychoacoustic auditory masking. The perturbations generated under different schemes have different signal forms and different naturalness.

[0004] Decibel distortion uses the intensity ratio between the perturbation and the original audio to indicate the audibility of the perturbation. By fixing the signal-to-noise ratio between the original audio and the perturbation at a larger value, the adversarial noise intensity is prevented from being too strong to be detected by humans. When decibel distortion is used as the audibility indicator, the adversarial perturbation generated is similar to white noise in signal morphology and auditory effect, which is quite different from normal human speech. This makes the generated adversarial samples less natural and easier to be distinguished by humans. In contrast, the psychoacoustic auditory masking scheme uses the theory of psychoacoustics, that is, for human hearing, high-energy sounds can mask low-energy sounds with similar frequencies and time points. The energy distribution of the perturbation generated with this as the audibility indicator is similar to that of the original audio in the time-frequency domain, so the perturbation sounds like a human whisper. Therefore, this type of adversarial sample has better naturalness, good concealment of the attack, and is difficult to be detected by humans, but it still retains some adversarial characteristics. In addition, the performance of adversarial samples generated under other audibility indicators such as absolute hearing threshold is between the above two.

[0005] Existing studies have not specifically described and analyzed the naturalness of adversarial audio, but they distinguish the differences in signal morphology between speech adversarial samples and benign speech by training classifiers, and distinguish between the two. Related work includes:

[0006] "Adversarial example detection by classification for deep speech recognition" published by SAMIZADE S et al. in the 2020 "Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)" uses inverted spectrum as input features and small-kernel convolutional neural network as classifier to detect adversarial noise in adversarial audio, and achieves detection of a white-box adversarial sample and a keyword recognition black-box adversarial sample.

[0007] CARLINI N et al. published "Hidden voice commands" in the "Proceedings of the USENIX security symposium" in 2016. They used the mean and standard deviation of short-term time-frequency domain features as input and the logistic regression model as a classifier to construct an adversarial sample detector that can detect speech adversarial samples generated based on the Mel-cepstral-quasi-cepstral method.

[0008] The "Audio Adversarial Sample Detection Method Based on Multi-Spectral Features" published by Ma Jian et al. in the "Journal of Dongguan University of Technology" in 2022 uses the time domain features and frequency domain features of adversarial samples as input, and trains convolutional neural networks as classifiers based on the features of multiple spectra to achieve detection of a white-box adversarial sample and a black-box adversarial sample.

[0009] The above work uses classifiers to distinguish adversarial samples from benign speech, but the proposed method does not analyze and utilize the unnaturalness of speech adversarial samples. Summary of the invention

[0010] In view of the above problems, the present invention proposes a method for evaluating the naturalness of speech against adversarial attacks on speech recognition models. First, five types of unnaturalness of adversarial samples are proposed, and for each type of unnaturalness, several acoustic-statistical features are proposed to describe such unnaturalness. Then, based on the above acoustic-statistical features, a naturalness index is designed to evaluate the naturalness of speech samples, which can quantify the similarity between the test speech and normal human speech.

[0011] The present invention is achieved through the following technical solutions:

[0012] A method for evaluating speech naturalness against speech adversarial attacks comprises the following steps:

[0013] Step S01, obtaining common features of the adversarial audio and screening to obtain common features related to unnaturalness, wherein the common features related to unnaturalness include five types, namely, discontinuous time domain signals, discontinuous frequency domain signals, irregular time domain signals, irregular frequency domain signals, and abnormal speech patterns;

[0014] Step S02, for each category of common features related to unnaturalness, calculating the acoustic-statistical features related to unnaturalness of the benign speech sample and the speech sample to be tested;

[0015] Step S03, designing a speech naturalness index according to the acoustic-statistical features related to unnaturalness of the benign speech sample, wherein the reference value of the speech naturalness index is the probability density distribution of the acoustic-statistical features related to unnaturalness of the benign speech sample;

[0016] Step S04, calculating the naturalness score of the speech sample to be tested according to the baseline value of the speech naturalness index. The lower the naturalness score, the greater the possibility of being attacked by speech adversarial attacks.

[0017] Furthermore, the unnaturalness-related acoustic-statistical features corresponding to the discontinuity of the time domain signal include the difference of peaks in the spectrum graph, the difference of resonance peaks in the spectrum envelope, the cosine distance of linear prediction coding, and the Euclidean distance of linear prediction coding.

[0018] Furthermore, the unnaturalness-related acoustic-statistical features corresponding to the discontinuity of the frequency domain signal include the height of the peak in the spectrum graph, the width of the peak in the spectrum graph, the kurtosis of the peak in the spectrum graph, the height of the resonance peak in the spectrum envelope, the width of the resonance peak in the spectrum envelope, the kurtosis of the resonance peak in the spectrum envelope, and the number of resonance peaks in the spectrum envelope.

[0019] Furthermore, the unnaturalness-related acoustic-statistical features corresponding to the irregularity of the time domain signal include the length of the linear prediction code, the autocorrelation of the linear prediction code, the zero-crossing rate of the linear prediction code, the period of the linear prediction code, and the zero-crossing distance of the linear prediction code.

[0020] Furthermore, the unnaturalness-related acoustic-statistical features corresponding to the irregularity of the frequency domain signal include the autocorrelation rate of the peaks in the spectrum graph, the autocorrelation rate of the spectrum amplitude in the spectrum graph, the zero-crossing rate of the spectrum amplitude in the spectrum graph, the period of the peaks in the spectrum graph, the period of the spectrum amplitude in the spectrum graph, and the zero-crossing distance of the spectrum amplitude in the spectrum graph.

[0021] Furthermore, the unnaturalness-related acoustic-statistical features corresponding to the abnormal speech pattern include the duration of phonemes and the interval time between characters.

[0022] Furthermore, the reference value of the speech naturalness index is expressed as:

[0023]

[0024] Among them, P X is the probability density distribution of the acoustic-statistical features related to unnaturalness of benign speech samples; is the probability density function of the jth acoustic-statistical feature in the benign speech sample. The probability density function is a set of statistics normalized from the acoustic-statistical features of the benign speech sample. , where M is the number of acoustic-statistical features, X is the set of benign speech samples, and x i is the i-th benign speech sample, d j is the jth acoustic-statistical feature, is the jth acoustic-statistical feature d of the speech sample j Normalized statistics.

[0025] Furthermore, the calculation formula for the naturalness score of the speech sample to be tested is:

[0026]

[0027] Where N is the naturalness score; the statistic after normalizing the acoustic-statistical features related to the unnaturalness of the speech sample x′ to be tested The probability density functions of the acoustic-statistical features of the benign speech sample are input, and the sum of the output results of the probability density functions of the acoustic-statistical features is used as the naturalness score.

[0028] The present invention has the following beneficial effects:

[0029] (1) The present invention measures the common features of adversarial audio under various attacks, and proposes features related to the unnaturalness of adversarial samples based on the causes of these common features and their connection with the principles of adversarial attacks. These features are summarized into five categories: discontinuous time domain signals, discontinuous frequency domain signals, irregular time domain signals, irregular frequency domain signals, and abnormal speech patterns. The present invention proposes the inherent properties of adversarial samples that exist under various attacks based on the unnaturalness of adversarial samples, which makes up for the lack of existing research in exploring the essential characteristics of speech adversarial samples and helps detect and defend against adversarial samples.

[0030] (2) The present invention designs acoustic-statistical features for describing speech unnaturalness, and based on this, proposes a speech naturalness index, which realizes the quantitative characterization of the inherent properties of speech adversarial samples. The probability that the test speech sample conforms to the benign audio distribution based on the above-mentioned acoustic-statistical features is calculated as a speech naturalness index. The speech naturalness index proposed in the present invention can evaluate the similarity between any speech sample and benign speech. In scenarios where machine learning models cannot be used, the naturalness score of an audio sample can be calculated quickly and easily. The possibility that the audio sample is an adversarial sample can be further obtained based on the naturalness score, thereby realizing preliminary detection of speech adversarial attacks. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a flowchart of a method for evaluating speech naturalness against speech adversarial attacks proposed by the present invention. DETAILED DESCRIPTION

[0032] The following are specific embodiments of the present invention and the accompanying drawings to further describe the technical solution of the present invention, but the present invention is not limited to these embodiments.

[0033] The accompanying drawings are merely schematic illustrations of the present invention. Some of the block diagrams shown in the accompanying drawings are functional entities, which do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and / or processor systems and / or microcontroller systems.

[0034] like Figure 1 The present invention proposes a method for evaluating the naturalness of speech against speech adversarial attacks, comprising:

[0035] Step S01, determine the common features related to the unnaturalness of the adversarial sample. By measuring the common features of the adversarial audio, the common features that are unique and common to the adversarial samples under various attacks are obtained. According to the causes of the common features and the connection with the adversarial attack principles, the features related to the unnaturalness of the adversarial samples are extracted from the common features and summarized into 5 categories, namely, discontinuous time domain signals, discontinuous frequency domain signals, irregular time domain signals, irregular frequency domain signals, and abnormal speech patterns.

[0036] In a specific implementation of the present invention, step S01 specifically includes:

[0037] Step S11, the time domain signal is discontinuous. Through the time domain continuity measurement experiment, it can be found that the continuity of the adversarial sample signal in the time domain is poor, that is, the signal changes drastically within tens of milliseconds, which is significantly different from human speech. Time domain continuity can be measured by the difference between the acoustic features of adjacent frames. The greater the difference, the worse the continuity. In this embodiment, the spectrum graph is used as an example to measure the continuity of the signal in the time domain. The acoustic feature of each frame in the spectrum graph is the spectrum amplitude of each frame signal. Based on the spectrum graph, the energy distribution difference between adjacent frames in the frequency domain can be calculated. By measuring the spectrum graph, it can be found that the difference in the spectrum amplitude of adjacent frames in the adversarial sample is large, while the difference in the spectrum amplitude of adjacent frames in the benign audio is small.

[0038] Step S12, the frequency domain signal is discontinuous. Through the frequency domain continuity measurement experiment, it can be found that the continuity of the adversarial sample signal in the frequency domain is poor, that is, the signal has a large change between adjacent frequency points, which is significantly different from human speech. Frequency domain continuity can be measured by the difference between acoustic features at adjacent frequency points. The acoustic features of discontinuous signals at adjacent frequency points are greatly different, that is, there are a large number of peaks in the feature vector along the spectrum direction. In this embodiment, the spectrum envelope is used as an example to measure the discontinuity of the signal in the frequency domain. The discontinuous signal is manifested in the spectrum envelope as a large number of narrow resonance peaks. By measuring the spectrum envelope, it can be found that compared with benign audio, the number of resonance peaks in the adversarial sample is large, the width is narrow, and the peak value is low, which is consistent with the characteristics of the signal being discontinuous in the frequency domain.

[0039] Step S13, the time domain signal is irregular. Through the time domain regularity measurement experiment, it can be found that the regularity of the adversarial sample signal in the time domain is poor, that is, the signal has poor periodicity in time series, which is significantly different from human speech. Time domain regularity can be measured by the stability of the time series related period over time. The more stable the period, the better the regularity. This embodiment takes the range of LPC period as an example to demonstrate the irregularity of the signal in the time domain. By measuring LPC, it can be found that in benign audio, as the LPC order increases, the LPC eigenvalue changes periodically and gradually approaches 0, and the period length of LPC is relatively stable, which shows that the time series of benign audio has the characteristics of periodic oscillation; however, in the adversarial sample, LPC does not have a fixed period length, which is consistent with the characteristics of irregular signals in the time domain.

[0040] Step S14, the frequency domain signal is irregular. Through the frequency domain regularity measurement experiment, it can be found that the regularity of the adversarial sample signal in the frequency domain is poor, that is, the signal has no obvious harmonics in the spectrum, which is significantly different from human speech. Frequency domain regularity can be measured by whether there are harmonics, because benign audio contains the fundamental and harmonic waves generated by human voice, which appear as periodic peaks in the spectrum. Therefore, it can be considered that benign audio has regularity in the frequency domain. This method takes the range of peak periods in the spectrum as an example to demonstrate the irregularity of signals in the frequency domain. By measuring the frequency difference between peaks in the spectrum, it can be found that benign audio has a stable period length in the spectrum diagram, while adversarial samples do not have a fixed period, which is consistent with the characteristics of irregular signals in the frequency domain.

[0041] Analysis of the causes of the above four types of discontinuity and irregularity. Through the above four steps, it can be found that the adversarial samples are discontinuous and irregular in the time domain and frequency domain, which is essentially different from benign speech. This is because the signals of the adversarial samples are not constrained by the physiological conditions of the human vocal organs. Specifically, in past studies, it was found that because the human vocal tract cannot change quickly, speech can only produce a small amount of changes within a time range of milliseconds, so it has continuity in the time domain. In addition, the frequency distribution of speech is limited by the frequency response of the vocal tract, and the energy gap between adjacent frequency points will not be too large, so speech is also continuous in the frequency domain. The periodicity in speech comes from the periodicity of the excitation signal. When humans make voiced sounds, the glottis will generate an excitation sound wave with a nearly periodic waveform, which makes the speech have a time correlation of fundamental waves, harmonics and near periods. In English, vowels and most consonants are voiced sounds, so it can be considered that there are periodic excitation sound waves most of the time when humans speak, which makes most human speech show regularity. However, adversarial samples are generated by machines, are not restricted by the glottis and vocal tract, and do not need to comply with the above constraints. In addition, since the data input into the speech recognition model is discretized audio, each discrete value is generated by a short-term narrow-band audio segment. It can be inferred that the continuity and regularity of the audio have limited effect on model recognition. Therefore, adversarial samples can also tamper with speech content without continuity and regularity. The above four adversarial features can reflect the unnaturalness of adversarial samples.

[0042] Step S15, the speech pattern is abnormal. Through the speech pattern measurement experiment, it can be found that the adversarial sample does not follow the language rules, that is, the duration and interval of the language unit in the speech recognition result of the adversarial sample are long. Benign audio is normal human speech, which can be expressed as a sequence of phonemes or characters. Since the speaking speed of human speech is relatively stable, the duration of phonemes and the interval between characters are relatively stable and will not fluctuate too much. However, the recognition results of the adversarial sample contain phonemes with very long duration and characters with very long intervals.

[0043] Analysis of the causes of abnormal speech patterns. The presence of such abnormal language patterns in adversarial samples may be because the language model in the speech recognition model does not consider the effects of speech unit duration and interval when correcting the recognition results. Therefore, adversarial samples lack language rule constraints when they are generated. This feature also reflects the unnaturalness of adversarial samples.

[0044] Step S02, designing acoustic-statistical features related to the unnaturalness of adversarial samples. For each unnaturalness of adversarial samples, an acoustic feature is designed to describe the unnaturalness, and further the overall distribution and peak distribution of the acoustic feature vector are characterized by statistical features. The acoustic features and statistical features are combined to form acoustic-statistical features.

[0045] In a specific implementation of the present invention, step S02 specifically includes:

[0046] Step S21, acoustic-statistical features for discontinuous time domain signals. In order to quantify the unnaturalness of the discontinuous time domain signals against audio, this embodiment designs four acoustic-statistical features: the difference of peaks in the spectrum graph, the difference of resonance peaks in the spectrum envelope, the cosine distance of linear predictive coding (LPC), and the Euclidean distance of LPC. Among them, the difference of peaks in the spectrum graph refers to the distance between peaks in adjacent frames of the spectrum, and the distance of the peaks is the frequency difference between the peaks. This feature can reflect the continuity of the signal in the time domain from the spectrum; the difference of resonance peaks in the spectrum envelope refers to the difference in the resonance peak frequencies of adjacent frames in the spectrum envelope; the cosine distance of LPC refers to the cosine distance between adjacent frame LPCs. LPC is the coefficient of the denominator polynomial when the vocal tract model is expressed by a polynomial fraction, so this feature can measure the difference between adjacent frames from the vocal tract model; the Euclidean distance of LPC refers to the Euclidean distance between adjacent frame LPCs.

[0047] Step S22, acoustic-statistical features for discontinuous frequency domain signals. In order to quantify the unnaturalness of the frequency domain signal discontinuity, this embodiment designs seven acoustic-statistical features: the height of the peak in the spectrum graph, the width of the peak in the spectrum graph, the kurtosis of the peak in the spectrum graph, the height of the resonance peak in the spectrum envelope, the width of the resonance peak in the spectrum envelope, the kurtosis of the resonance peak in the spectrum envelope, and the number of resonance peaks in the spectrum envelope. Among them, the height of the peak in the spectrum graph refers to the numerical value of the peak; the width of the peak in the spectrum graph refers to the size of the frequency band covered by the peak; the kurtosis of the peak in the spectrum graph refers to the ratio of the height to the width of the peak; the resonance peak in the spectrum envelope is the peak in the spectrum envelope, the resonance peak height refers to the numerical value of the spectrum envelope where the resonance peak is located, the resonance peak width refers to the size of the frequency band covered by the resonance peak, the resonance peak kurtosis refers to the ratio of the height to the width of the resonance peak, and the number of resonance peaks refers to the average number of resonance peaks per frame in the audio.

[0048] Step S23, for the acoustic-statistical features of irregular time domain signals. In order to quantify the irregularity of time domain signals to combat the unnaturalness of audio, this embodiment designs five acoustic-statistical features: LPC length, LPC autocorrelation, LPC zero crossing rate, LPC period, and LPC zero crossing distance. LPC is the coefficient of each state term when the vocal tract model is represented by a state equation. Specifically, the vocal tract model is represented by a state equation as follows:

[0049]

[0050] Among them, x(t) is the amplitude of the signal at the tth moment, a i is the i-th order LPC. Therefore, the k-order LPC represents the correlation between the signal sampling value at one moment and the signal sampling values ​​at the previous k moments. The size of the LPC value can evaluate the strength of the timing correlation at each moment to determine the timing length with strong correlation. Here, the timing length with strong correlation is called the length of LPC. The autocorrelation, zero-crossing rate, period, and zero-crossing distance of LPC refer to the autocorrelation, zero-crossing rate, period, and distance between adjacent zero-crossing points of the sequence composed of LPC values.

[0051] Step S24, for the acoustic-statistical features of the irregular frequency domain signal. In order to quantify the unnaturalness of the irregular frequency domain signal against the audio, this embodiment designs six acoustic-statistical features: the autocorrelation rate of the peak in the spectrum graph, the autocorrelation rate of the spectrum amplitude in the spectrum graph, the zero-crossing rate of the spectrum amplitude in the spectrum graph, the period of the peak in the spectrum graph, the period of the spectrum amplitude in the spectrum graph, and the zero-crossing point distance of the spectrum amplitude in the spectrum graph. Among them, the autocorrelation rate of the peak in the spectrum graph and the period of the peak in the spectrum graph refer to the autocorrelation and period length of the frequency point sequence of the peak in the spectrum graph; the autocorrelation rate of the spectrum amplitude in the spectrum graph, the zero-crossing rate of the spectrum amplitude in the spectrum graph, the period of the spectrum amplitude in the spectrum graph, and the zero-crossing point distance of the spectrum amplitude in the spectrum graph refer to the autocorrelation, zero-crossing rate, period length and distance between adjacent zero-crossing points of the spectrum amplitude sequence.

[0052] Step S25, acoustic-statistical features for abnormal speech patterns. In order to quantify the unnaturalness of the abnormal speech patterns against the audio, this embodiment designs two acoustic-statistical features: the duration of the phoneme and the interval time of the characters. The phoneme duration refers to the duration of the pronunciation of a single phoneme in a word, and the character interval time refers to the time difference between the midpoints of the duration of the pronunciation of two adjacent characters in the same word.

[0053] Step S03, designing a speech naturalness index according to the acoustic-statistical features related to unnaturalness of benign speech samples, wherein the reference value of the speech naturalness index is the probability density distribution of the acoustic-statistical features related to unnaturalness of benign speech samples.

[0054] In a specific implementation of the present invention, step S03 specifically includes:

[0055] Step S31, naturalness definition. In this embodiment, the naturalness score is defined as the probability that the test speech sample conforms to the benign audio distribution on the above 24 acoustic-statistical features. Since the characteristic value of benign audio is a distribution rather than a single value, it is difficult to directly calculate the distance between the characteristic value of the test speech sample and the characteristic value of the benign audio. Therefore, when calculating the naturalness score, the probability that the test speech sample belongs to benign audio is calculated for each acoustic-statistical feature, and the sum of the probabilities of the 24 acoustic-statistical features is used to represent the similarity between the measured sample and the benign sample.

[0056] Step S32, the benchmark value of the naturalness score. First, the probability density distribution of the above 24 acoustic-statistical features in the benign audio needs to be calculated, and this distribution is used as the benchmark value for calculating the naturalness score, which is recorded as:

[0057]

[0058] in, is the probability density function of the jth acoustic-statistical feature in a large number of benign audio. This probability density function is derived from the data It is obtained by calculation in , which is the set of normalized statistics of benign audio acoustic-statistical features, X is the set of benign audio, x i is the i-th benign audio, d j is the jth acoustic-statistical feature, is the jth acoustic-statistical feature d of benign audio j Normalized statistics.

[0059] Step S04, calculate the naturalness score of the speech sample to be tested according to the reference value of the speech naturalness index. The lower the naturalness score, the greater the possibility of being attacked by speech adversarial attacks. Specifically, for the test speech sample x', it is necessary to calculate the above 24 acoustic-statistical features, normalize the feature vector or matrix, and calculate the statistics to obtain the set:

[0060]

[0061] Enter the data in the collection into P X And the sum of the outputs is taken as the naturalness score N:

[0062]

[0063] For the test speech sample, the lower the naturalness score, the greater the distinction from benign audio, and the more likely it is an adversarial sample.

[0064] The present invention proposes a method for evaluating the naturalness of speech against speech adversarial attacks. Aiming at five common features related to unnaturalness, an acoustic-statistical feature for describing unnaturalness is designed, thereby achieving quantitative characterization of the inherent properties of speech adversarial samples. Based on this, a speech naturalness index is proposed, which can evaluate the similarity between any speech sample and benign speech, and complete the naturalness evaluation of any speech sample. In scenarios where machine learning models cannot be used, the naturalness of an audio sample can be calculated quickly and easily, and the possibility of the audio sample being an adversarial sample can also be judged based on the naturalness score, which has high application value.

[0065] The above examples are only specific embodiments of the present invention. Obviously, the present invention is not limited to the above examples, and many variations are possible. All variations that can be directly derived or associated with the contents disclosed by a person skilled in the art should be considered as the protection scope of the present invention.

Claims

1. A method for evaluating speech naturalness against speech adversarial attacks, characterized in that: The following steps are involved: Step S01, obtaining common features of the adversarial audio and screening to obtain common features related to unnaturalness, wherein the common features related to unnaturalness include five types, namely, discontinuous time domain signals, discontinuous frequency domain signals, irregular time domain signals, irregular frequency domain signals, and abnormal speech patterns; Step S02, for each category of common features related to unnaturalness, calculating the acoustic-statistical features related to unnaturalness of the benign speech sample and the speech sample to be tested; Step S03, designing a speech naturalness index according to the acoustic-statistical features related to unnaturalness of the benign speech sample, wherein the reference value of the speech naturalness index is the probability density distribution of the acoustic-statistical features related to unnaturalness of the benign speech sample; Step S04, calculating the naturalness score of the speech sample to be tested according to the baseline value of the speech naturalness index. The lower the naturalness score, the greater the possibility of being attacked by speech adversarial attacks.

2. The method for evaluating speech naturalness against speech adversarial attacks according to claim 1, characterized in that: The unnaturalness-related acoustic-statistical features corresponding to the discontinuity of the time domain signal include the difference of peaks in the spectrum diagram, the difference of resonance peaks in the spectrum envelope, the cosine distance of linear prediction coding, and the Euclidean distance of linear prediction coding.

3. The method for evaluating speech naturalness against speech adversarial attacks according to claim 1, characterized in that: The unnaturalness-related acoustic-statistical features corresponding to the discontinuity of the frequency domain signal include the height of the peak in the spectrum graph, the width of the peak in the spectrum graph, the kurtosis of the peak in the spectrum graph, the height of the resonance peak in the spectrum envelope, the width of the resonance peak in the spectrum envelope, the kurtosis of the resonance peak in the spectrum envelope, and the number of resonance peaks in the spectrum envelope.

4. The method for evaluating speech naturalness against speech adversarial attacks according to claim 1, characterized in that: The unnaturalness-related acoustic-statistical features corresponding to the irregularity of the time domain signal include the length of the linear prediction code, the autocorrelation of the linear prediction code, the zero-crossing rate of the linear prediction code, the period of the linear prediction code, and the zero-crossing distance of the linear prediction code.

5. The method for evaluating speech naturalness against speech adversarial attacks according to claim 1, characterized in that: The unnaturalness-related acoustic-statistical features corresponding to the irregularity of the frequency domain signal include the autocorrelation rate of the peaks in the spectrum graph, the autocorrelation rate of the spectrum amplitude in the spectrum graph, the zero-crossing rate of the spectrum amplitude in the spectrum graph, the period of the peaks in the spectrum graph, the period of the spectrum amplitude in the spectrum graph, and the zero-crossing distance of the spectrum amplitude in the spectrum graph.

6. The method for evaluating speech naturalness against speech adversarial attacks according to claim 1, characterized in that: The unnaturalness-related acoustic-statistical features corresponding to the abnormal speech pattern include the duration of phonemes and the interval time between characters.

7. The method for evaluating speech naturalness against speech adversarial attacks according to claim 1, characterized in that: The reference value of the speech naturalness index is expressed as: Among them, P X is the probability density distribution of the acoustic-statistical features related to unnaturalness of benign speech samples; is the probability density function of the jth acoustic-statistical feature in the benign speech sample. The probability density function is a set of statistics normalized from the acoustic-statistical features of the benign speech sample. , where M is the number of acoustic-statistical features, X is the set of benign speech samples, and x i is the i-th benign speech sample, d j is the jth acoustic-statistical feature, is the speech sample x i The jth acoustic-statistical feature d j Normalized statistics.

8. The method for evaluating speech naturalness against speech adversarial attacks according to claim 7, characterized in that: The calculation formula for the naturalness score of the speech sample to be tested is: Where N is the naturalness score; the statistic after normalizing the acoustic-statistical features related to the unnaturalness of the speech sample x′ to be tested The probability density functions of the acoustic-statistical features of the benign speech sample are input, and the sum of the output results of the probability density functions of the acoustic-statistical features is used as the naturalness score.

Images