Nonlinear injection attack detection method and device based on hardware characteristics

Abstract

The application discloses a kind of nonlinear injection attack detection method and device based on hardware characteristics, wherein, detection method includes the following steps: (1) the speech activity detection is carried out to the audio to be measured collected, and the audio to be measured is cut according to speech part, and a plurality of speech segments are obtained after eliminating non-speech part;(2) for each speech segment, simultaneously carry out undersampling audio detection and abnormal white noise detection;If there is undersampling audio similar to normal speech part and / or there is approximate white noise highly correlated with speech energy, it is determined that the speech segment is nonlinear injection, and a warning is issued to the user.The detection method in the application can be directly deployed on a smart device, and the detection device can be deployed near the smart device, which can independently complete the detection work and provide a convenient, universal and unavoidable nonlinear injection attack detection scheme for voice assistant users.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of voice assistant security, and particularly relates to a nonlinear injection attack detection method and device based on hardware characteristics. BACKGROUND

[0002] Personal voice assistants are widely deployed in various smart devices, such as mobile phones, smart speakers, smart homes, etc. With the popularity of smart devices, more and more users begin to use voice assistants. Personal voice assistants bring convenience to users, but also bring new risks. Guoming Zhang et al. proposed an attack method of injecting malicious instructions into voice assistants using ultrasonic waves in "DolphinAttack: Inaudible Voice Commands". The attack method takes advantage of the nonlinearity of the recording module of the smart device, and generates a malicious instruction corresponding to the human voice frequency band signal in the recording process by modulating the ultrasonic signal, which is called nonlinear injection attack. Nonlinear injection attack can inject malicious instructions such as opening the garage door and sending messages into the voice assistant without the user's knowledge (the human ear cannot perceive ultrasonic waves), which poses a huge security risk to users.

[0003] Recently, researchers have proposed several detection and defense works against nonlinear injection attacks. For example, Nirupam Roy et al. proposed a defense mechanism to detect the audio features caused by nonlinear injection in "Inaudible Voice Commands: The Long-Range Attack and Defense". During the ultrasonic injection process, abnormal features such as positive amplitude spectrum, excessive low-frequency energy below 50Hz, and time-domain correlation with speech signals will be generated. This method detects the above features to determine whether an audio is injected by ultrasonic nonlinear injection. This method achieves high detection accuracy for traditional nonlinear injection attacks, but subsequent studies show that attackers can design modulated ultrasonic signals to avoid the generation of the above audio features, thereby evading detection.

[0004] Yitao He et al. in "Canceling Inaudible Voice Commands Against Voice Control Systems" use additional ultrasonic probes to assist in detection. These ultrasonic probes are deployed near the smart device and continuously play ultrasonic signals of a specific frequency. When an attacker performs a nonlinear injection attack, the ultrasonic waves not only produce malicious commands designed by the attacker in the recording module, but also combine with the defense ultrasonic signals to produce audio signals related to the malicious commands. The defense system detects nonlinear injection attacks by detecting whether there is a specific correlation signal in a segment of voice signal. This defense method cannot be circumvented by attackers, but it requires the deployment of additional hardware to continuously play ultrasonic waves, reducing the portability and practicality of the method, and posing a health threat to users.

[0005] Guoming Zhang et al. in "EarArray: Defending against DolphinAttack via Acoustic Attenuation" propose a defense method that can detect nonlinear injection attacks on multi-microphone devices. Since the energy attenuation of ultrasonic waves in the air is significantly faster than that of conventional audio, this method detects nonlinear injection attacks by comparing the audio energy received by different microphones. If the energy difference between the signals received by different microphones is greater than a set threshold, the audio is a nonlinear attack injection. This method is only applicable to multi-microphone systems, and attackers can circumvent detection by designing ultrasonic probe positions so that the ultrasonic waves reach each microphone with the same degree of attenuation. SUMMARY

[0006] The present application provides a nonlinear injection attack detection method and device based on hardware characteristics, which can provide a universal, convenient, and non-circumventable nonlinear injection detection scheme for voice assistant users.

[0007] A nonlinear injection attack detection method based on hardware characteristics, comprising the following steps:

[0008] (1) Perform voice activity detection on the collected audio to be tested, and divide the audio to be tested into speech parts. After removing the non-speech part, a plurality of speech segments are obtained;

[0009] (2) For each speech segment, simultaneously perform undersampling audio detection and abnormal white noise detection; if there is undersampling audio similar to the normal speech part and / or there is approximately white noise highly correlated with the speech energy, it is determined that the speech segment is nonlinear injection, and a warning is issued to the user.

[0010] The nonlinear injection attack detection method can be directly deployed on the intelligent device without changing the hardware basis to detect the nonlinear injection attack.

[0011] Further, in step (2), the specific process of the under-sampling audio detection is as follows:

[0012] (2-1) performing short-time Fourier transform (STFT) on the voice segment to extract the time-frequency spectrum of the audio signal;

[0013] (2-2) locating the single-frequency signal in the time-frequency spectrum according to the energy, and traversing the single-frequency signals;

[0014] (2-3) for each single-frequency signal, selecting a frequency band signal with a bandwidth of an adjustable parameter b near the single-frequency signal, performing flip processing on the signal below the single-frequency signal, and respectively calculating the similarity with the voice signal starting from 0 Hz;

[0015] If the similarity of one side of the upper and lower signals is higher than the threshold, it is determined that the audio is injected by the nonlinear injection; if all the signals near the single-frequency are lower than the threshold, the audio is a regular audio.

[0016] Preferably, the value of the adjustable parameter b is 2K-4K.

[0017] In step (2), the specific process of the abnormal white noise detection is as follows:

[0018] (2-1') performing short-time Fourier transform (STFT) on the voice segment to extract the time-frequency spectrum of the audio signal;

[0019] (2-2') selecting time points at which the energy of the voice part changes sharply, respectively calculating the energy of the voice part and the energy of the high-frequency part at each time point, and respectively forming energy matrices;

[0020] (2-3') calculating the similarity between the two energy matrices, if the similarity is higher than the threshold, the audio part is a nonlinear injection, otherwise it is a regular audio.

[0021] Further, in step (2-2'), the voice part refers to the part from 0 Hz to c Hz, and c is an adjustable parameter; the high-frequency part refers to the part from d Hz to 24 KHz, and d is an adjustable parameter.

[0022] Preferably, the value of the adjustable parameter c is 2K-4K, and the value of the adjustable parameter d is 16K-18K.

[0023] The application also provides a nonlinear injection attack detection device based on hardware characteristics, which comprises an Arduino UNO development board and a microphone module and a memory card module connected with the Arduino UNO development board; in the detection process, the Arduino UNO development board controls the recording sampling rate of the microphone module through programming, and records the audio to be detected with different sampling rates respectively; then the frequency bands of the audio on the time-frequency spectrum are compared, if the frequency bands of the recorded audio under two kinds of sampling rates are consistent, it is a regular audio, otherwise it is a nonlinear injection.

[0024] The detection device of the application can be deployed near the smart device to complete the detection work independently by combining software and hardware.

[0025] Compared with the prior art, the application has the following beneficial effects:

[0026] The application designs a nonlinear injection attack detection method which can be deployed in the smart device to be protected and a nonlinear injection attack detection device which can be deployed near the smart device, both of which can complete the detection work independently, and can realize convenient, universal and non-avoidable nonlinear injection attack detection. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1 It is a structure diagram of a regular recording module and a nonlinear injection attack process;

[0028] Figure 2 It is a signal diagram obtained by each component of the regular recording module in the nonlinear injection attack process;

[0029] Figure 3 It is an audio schematic diagram after undersampling;

[0030] Figure 4 It is an approximate white noise schematic diagram during nonlinear injection;

[0031] Figure 5 It is a system block diagram of the nonlinear injection attack detection method and the detection device of the application;

[0032] Figure 6 It is a model diagram of the nonlinear injection attack detection device of the application;

[0033] Figure 7 It is a time-frequency spectrum of ultrasonic voice instructions and regular voice instructions recorded by the detection device of the application with different sampling rates;

[0034] Figure 8 The result graph of the attack detection of the detection method of the application on 6 smart phones;

[0035] Figure 9 The detection effect graph of the detection method of the application under different attack distances, different carrier frequencies and different environmental noises. DETAILED DESCRIPTION

[0036] The application will be further described in detail below in combination with the drawings and examples, and it should be pointed out that the following examples are intended to facilitate the understanding of the application and do not limit the application in any way.

[0037] The emergence of nonlinear injection attacks brings huge security risks to voice assistant users, and existing defense detection methods have defects such as being evadable, limited application scenarios, and harming user health. The present application aims to provide a universal, convenient and evadable nonlinear injection detection scheme for voice assistant users.

[0038] The application designs a detection scheme based on the hardware characteristics of the recording device during nonlinear injection.

[0039] The core principle of nonlinear injection attack is to use the nonlinearity of the recording device to convert ultrasonic waves into normal frequency audio. The conventional recording module includes a microphone (Mic), an amplifier (Amplifier), a low-pass filter (LPF) and an analog-to-digital converter (ADC), as shown in Figure 1 The human ear cannot perceive audio above 20KHz, and this threshold will further decrease with age. Therefore, a low-pass filter is deployed in the conventional recording module to filter out signals above 20KHz that are inaudible to humans to improve recording quality.

[0040] However, research shows that there is a certain nonlinearity in each module of the recording module when transmitting signals above 23KHz, with the amplifier being the most obvious. Specifically, when an audio signal x(t) passes through the amplifier, the signal is transformed into y AMP (t)=A1x(t)+A2x(t) 2 , where A1 is the amplifier amplification coefficient and A2 is the nonlinearity coefficient. Using this characteristic, the attacker can design an ultrasonic signal:

[0041] Sig attack =(1+m(t))cos(ω c t)

[0042] , where m(t) is a conventional audio signal, i.e. the malicious instruction that the attacker wants the voice assistant to receive; cos(ω c t) is an ultrasonic carrier signal with a frequency of ω cFor values greater than 23K, such as 40K.

[0043] After the amplifier, the attack signal becomes:

[0044]

[0045] The low-pass filter will contain ω c and filter out the high-frequency components of 2ω c , the attack signal becomes:

[0046]

[0047] At this time, the attack signal already contains m(t), that is, the attacker successfully injects malicious instructions into the recording system. The nonlinear injection attack process can be seen in Figure 1 .

[0048] The above is the case of nonlinear injection theory attack, and in actual experiments, it is found that there are some defects in the components of the recording module, which will leave traces that cannot be eliminated in the nonlinear injection attack process.

[0049] First, the low-pass filter of some devices has defects and cannot perfectly filter out high-frequency signals. At this time, the signal obtained through the analog-to-digital converter contains not only malicious instructions but also high-frequency signals that are undersampled to low frequencies, as shown in Figure 2 .

[0050] The role of the analog-to-digital converter is to convert continuous analog electrical signals into digital signals at a certain sampling rate. According to the sampling theorem, the highest frequency of the sampled signal is half of the sampling frequency. Currently, smart devices (using a smart phone as an example) that carry voice assistants often use a recording sampling rate of 48KHz. Therefore, high-frequency signals that are not filtered out by the low-pass filter will be undersampled to below 24KHz. For example, the attacker modulates the malicious instructions to 40KHz and plays them to the target phone (with a defective low-pass filter). The target phone will receive not only the regular voice starting at 0Hz but also an audio signal starting at 8KHz, which is highly similar to the regular voice starting at 0Hz, as well as an 8KHz single-frequency signal (generated by undersampling the ultrasonic carrier signal), as shown in Figure 3 The rectangular box part is the audio after undersampling.

[0051] Secondly, due to the small value of the quadratic term coefficient A2 in the nonlinear effect, the attacker needs to increase the energy of the ultrasonic wave to achieve better attack effect. Due to the hardware defects of the recording module, most smart devices (using a smart phone as an example) will generate noise in the full frequency band when receiving high-energy ultrasonic waves. This noise is approximately white noise in the frequency spectrum and is highly correlated with the energy of the ultrasonic wave. As shown in Figure 4 , the energy of this approximate white noise is highly correlated with the energy of the injected voice during nonlinear injection.

[0052] Based on the above findings, the present application designs a set of nonlinear injection attack detection scheme, the system block diagram is shown in Figure 5 The detection scheme contains pure software detection method and hardware and software combined detection device, both of which can independently complete the detection work.

[0053] The pure software detection method can be deployed on existing intelligent devices to detect nonlinear injection attacks without changing the hardware basis. This method detects nonlinear injection attacks by detecting whether there is an undersampled audio similar to the regular speech part and whether there is an approximate white noise highly correlated with the speech energy.

[0054] Specifically, the detection method first performs voice activity detection (VAD) on the audio to be tested and cuts off the non-speech part. For each speech segment, the method simultaneously performs undersampled audio detection and abnormal white noise detection. If one of the detections is abnormal, it is determined that the speech is nonlinearly injected, and a warning is issued to the user.

[0055] The undersampled audio detection algorithm is as follows:

[0056]

[0057] First, the time-domain audio signal is subjected to short-time Fourier transform (STFT) to extract the time-frequency spectrum of the audio signal. Second, the single-frequency signals in the time-frequency spectrum are located by energy, and these single-frequency signals are traversed. For each single-frequency signal, the detection algorithm selects the frequency band signal with a bandwidth of b (a tunable parameter) near it (where the signal below the single frequency needs to be flipped), and respectively calculates the similarity with the speech signal starting from 0 Hz. If the similarity of one side of the upper and lower signals is higher than the threshold, it is determined that the audio is nonlinearly injected. If all the signals near the single frequency are below the threshold, the audio is regular audio.

[0058] The abnormal white noise detection algorithm is as follows:

[0059]

[0060] First, the time-domain audio signal is subjected to short-time Fourier transform (STFT) to extract the time-frequency spectrum of the audio signal. Second, the time points with high energy changes in the speech part are selected, the energy of the speech part (i.e. 0Hz to cHz part, c is a tunable parameter) and the energy of the high frequency part (i.e. dHz to 24KHz part, d is a tunable parameter) at each time point are calculated respectively, and an energy matrix is formed. Calculate the similarity between the two matrices. If it is higher than the threshold, the audio in this part is nonlinearly injected, otherwise it is regular audio. Since the high frequency background noise in daily life is relatively stable, selecting the high frequency part for calculation has higher accuracy.

[0061] The hardware and software combined detection device is designed based on the principle of under-sampling, and a variable sampling rate recording module is used for nonlinear injection attack detection. The detection module prototype is constructed by using an Arduino UNO development board and corresponding microphone modules and storage card modules, as shown in Figure 6 The Arduino UNO development board can realize variable recording sampling rate through programming. During detection, different sampling rates are used for recording (for example, 14KHz and 15KHz), and then the frequency bands of the audio on the time-frequency spectrum are compared. If the frequency bands of the audio recorded under the two sampling rates are consistent, it is a regular audio, otherwise it is a nonlinear injection, as shown in Figure 7 .

[0062] To verify the fixed effect of the present application, a large-scale evaluation experiment is designed to verify the effectiveness of the pure software detection method. The present application uses 6 smartphones as attack targets to carry out experiments, and performs nonlinear injection attack and plays regular audio (195 pieces each) on them respectively, and sends the recorded audio into the detection algorithm for detection. The overall results are shown in Figure 8 , in which (a) is the overall detection accuracy, (b) is the overall true positive / negative rate, and (c) is the overall ROC curve.

[0063] At the same time, the detection effect of the detection method under different attack distances, different carrier frequencies and different environmental noises is also tested, as shown in Figure 9 , in which (a) represents the distance influence, (b) represents the carrier frequency influence, and (c) represents the environmental noise influence. The experimental results show that the detection algorithm has a detection accuracy close to 100%, and has strong robustness under different interferences.

[0064] The above embodiments describe the technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only a specific embodiment of the present application and is not used to limit the present application. Any modification, supplement and equivalent replacement made within the principle range of the present application shall be included in the protection scope of the present application.

Claims

1. A hardware feature-based nonlinear injection attack detection method, characterized in that, The nonlinear injection attack detection method is directly deployed on the smart device, and comprises the following steps: (1) performing voice activity detection on the collected audio to be tested, and cutting the audio to be tested according to the voice part to obtain a plurality of voice segments after eliminating the non-voice part; (2) simultaneously performing undersampling audio detection and abnormal white noise detection on each voice segment; if there is undersampling audio similar to the normal voice part and / or there is approximate white noise highly correlated with the voice energy, it is determined that the voice segment is subjected to nonlinear injection, and a warning is issued to the user; The specific process of performing undersampling audio detection is as follows: (2-1) performing short-time Fourier transform (STFT) on the voice segment to extract the time-frequency spectrum of the audio signal; (2-2) locating the single-frequency signal in the time-frequency spectrum according to the energy, and traversing these single-frequency signals; (2-3) for each single-frequency signal, selecting a frequency band signal with a bandwidth of adjustable parameter b near the single-frequency signal, performing flip processing on the signal below the single-frequency, and performing similarity calculation on the signal with the 0Hz starting point voice signal respectively; If the similarity of one side of the upper and lower signals is higher than the threshold, it is determined that the voice segment is subjected to nonlinear injection; if all single-frequency nearby signals are lower than the threshold, the voice segment is normal audio; The specific process of performing abnormal white noise detection is as follows: (2-1') performing short-time Fourier transform (STFT) on the voice segment to extract the time-frequency spectrum of the audio signal; (2-2') selecting time points with sharp energy changes in the voice part, calculating the energy of the voice part and the energy of the high-frequency part at each time point respectively, and forming an energy matrix respectively; (2-3') calculating the similarity between the two energy matrices, if higher than the threshold, the voice segment is nonlinear injection, otherwise it is normal audio.

2. The hardware feature based nonlinear injection attack detection method of claim 1, wherein, The value of the adjustable parameter b is 2K-4K.

3. The hardware feature based nonlinear injection attack detection method of claim 1, wherein, In step (2-2'), the voice part refers to the 0Hz-cHz part, and c is an adjustable parameter; the high-frequency part refers to the dHz-24KHz part, and d is an adjustable parameter.

4. The hardware feature based nonlinear injection attack detection method of claim 3, wherein, The value of the adjustable parameter c is 2K-4K, and the value of the adjustable parameter d is 16K-18K.

5. A hardware characteristic based nonlinear injection attack detection apparatus, characterized by, The Arduino UNO development board, the microphone module connected with the Arduino UNO development board, and the storage card module are included. During the detection process, the Arduino UNO development board controls the recording sampling rate of the microphone module through programming, and records the audio to be tested at different sampling rates; Then compare the frequency bands of the audio on the time-frequency spectrum, if the frequency bands of the recorded audio at two different sampling rates are consistent, it is normal audio, otherwise it is nonlinear injection.

Images