A Microphone Array-Based Method for Ultrasonic Attack Detection and Localization

Abstract

This invention discloses a method for detecting and locating ultrasonic attacks based on a microphone array, relating to the field of ultrasonic attack defense. The method includes the following steps: S1: An ultrasonic speaker is placed in the microphone array to periodically broadcast an auxiliary signal, which is down-converted to a possible ultrasonic attack signal, thereby monitoring for the presence of an ultrasonic attack; S2: The collected data is preprocessed by filtering, denoising, segmenting, and standardizing the signals received by the microphones. This invention proposes a method for detecting and locating ultrasonic attacks based on a microphone array, widely used in smart devices, and an ultrasonic speaker. This technology can not only detect ultrasonic attack events promptly and efficiently, but also accurately measure the attacker's location in the first instance, thus eliminating the threat for the user in a timely manner.

Description

Technical Field

[0001] This invention relates to the field of ultrasonic attack defense, and in particular to a method for ultrasonic attack detection and localization based on a microphone array. Background Technology

[0002] Ultrasonic attacks based on the nonlinear effects of voice devices have become a significant security threat to voice control systems. Ultrasonic voice commands are typically generated by modulating ultrasonic signals of 40kHz or higher with human or synthesized speech, and the microphone amplifier exhibits nonlinear effects. ] This attack redirects ultrasonic signals to lower frequencies within the passband, allowing commands to be recorded as normal voice, thus enabling the attacker to control the target device's voice assistant to perform various tasks without the user's knowledge. This type of attack poses a serious threat to voice control applications, and voice control systems can be used to disable home security systems. Attackers could instruct the user's device to send fake text messages and emails, post fake online posts, add fake events to the calendar, or even deny internet services. Attackers could also input commands to activate airplane mode or disconnect all wireless communications.

[0003] Current detection methods for this type of attack mostly require complex time-frequency calculations and have low robustness to environmental noise, making them easily fooled by attackers. More importantly, current defense systems are unable to effectively counterattack attackers, and traditional localization methods are difficult to apply directly to cross-frequency band scenarios. There is still significant room for development in defense against voice attacks caused by nonlinear effects on voice devices. Summary of the Invention

[0004] The purpose of this invention is to provide a method for detecting and locating ultrasonic attacks based on a microphone array. This method utilizes a microphone array, widely used in smart devices, and an ultrasonic speaker. The ultrasonic speaker periodically broadcasts auxiliary signals within the microphone array. This technology can not only detect ultrasonic attack events promptly and efficiently, but also accurately measure the attacker's location in the first instance, thereby eliminating threats for users in a timely manner and enabling timely countermeasures to protect information security and property security, thus solving the problems mentioned in the background section.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a method for detecting and locating ultrasonic attacks based on a microphone array, comprising the following steps:

[0006] S1: Set an ultrasonic speaker in the microphone array to periodically broadcast an auxiliary signal, which is down-converted to a possible ultrasonic attack signal, thereby monitoring for the presence of ultrasonic attacks.

[0007] S2: Preprocess the acquired data by filtering, denoising, segmenting, and standardizing the signals received by the microphone;

[0008] S3: Detect the preprocessed signal, perform spectrum peak detection on the difference frequency signal, find outliers, and then perform template matching with the auxiliary signal;

[0009] S4: Locate the ultrasonic attack source, extract the time delay between each microphone pair as a feature, and input it into the SVM classifier for localization.

[0010] S5: Output results. The system indicates whether an ultrasonic attack exists and the location of the ultrasonic attack source.

[0011] Preferably, step S1 includes the following steps:

[0012] S101: An ultrasonic loudspeaker with multiple frequency broadcasts is placed in the microphone array to periodically broadcast an auxiliary signal. This signal is then down-converted with an attack signal of any frequency under nonlinear action, generating a difference frequency signal within the range of [10kHz, 20kHz], which can then be detected.

[0013] Sg = cos(2πf1t + 2πf2t + 2πf3t) (1),

[0014] Sg is the auxiliary signal, and f1, f2, and f3 are the three frequencies of the multi-tone signal, which are 25kHz, 45kHz, and 65kHz, respectively.

[0015] S102: Encode the auxiliary signal to reduce the interference of environmental noise in subsequent signal analysis.

[0016] Preferably, step S2 includes the following specific steps:

[0017] S201: Performs noise reduction processing on the multi-channel data acquired by the microphone array, mainly including low-pass filtering and smoothing filtering;

[0018] S202: Standardize the data, mainly including segmentation and Z-Score standardization.

[0019] Preferably, the specific operation process of step S201 is as follows:

[0020] Use a bandpass filter with a cutoff frequency of 10-20kHz to remove frequency band interference outside the difference frequency signal;

[0021] Using a moving average filter, outliers and glitches are removed, i.e., high-frequency noise is removed to smooth the signal.

[0022] Preferably, step S3 includes the following specific steps:

[0023] S301: Detect whether there is a generated difference frequency signal in the audio signal. If there is a difference frequency signal, determine the location of the difference frequency signal and proceed to step S302. If no difference frequency signal is detected, the system outputs "no attack".

[0024] S302: Perform spectrum value detection on the difference frequency signal, find outliers, and use a cross-correlation algorithm to match the difference frequency signal with the auxiliary signal to determine whether it is an ultrasonic attack signal.

[0025] Preferably, the specific implementation process of step S301 is as follows:

[0026] The attack signal is set as SA=(m(t)+ɑ)cos(2fat)(2),

[0027] Where (m(t)+ɑ) is the illegal voice command modulated by the attacker, and fa is the carrier signal frequency used by the attacker;

[0028] The attack signal and the auxiliary signal broadcast by the ultrasonic loudspeaker undergo down-conversion. Substituting this into formula (1), the following signal is generated: SM=(m(t)+α)cos(2π(fa-fg)t)(3),

[0029] Perform FFT transformation on the down-conversion signal, that is, process formula (3), perform peak detection within the preset frequency band, and determine whether there is an ultrasonic attack signal. If there is no ultrasonic attack signal, the system outputs no attack; if there is an ultrasonic attack signal, proceed to step S302.

[0030] Preferably, step S4 includes the following specific steps:

[0031] S401: Locates ultrasonic attack signals, extracts features from difference frequency signals, performs cross-correlation calculations on each microphone pair, and obtains the time delay difference (TDOA) between signals;

[0032] S402: Centered on the microphone array with a radius of 0.5m, it collects data from different locations of the attack source and establishes a database;

[0033] S403: Use an SVM classifier for localization and feed the results back to the system.

[0034] Preferably, the data at different locations in step S402 includes the specific values ​​of the horizontal and vertical angles of the attack source.

[0035] Preferably, step S5 includes the following steps: the system indicates whether an ultrasonic attack exists; if an attack is detected, it continues to output the ultrasonic attack location result composed of horizontal and vertical angles; if no ultrasonic attack is detected, it outputs "no attack".

[0036] Compared with the prior art, the beneficial effects of the present invention are:

[0037] 1. The present invention proposes an ultrasonic attack detection and localization method based on a microphone array. It uses a microphone array, which is widely used in smart devices, and an ultrasonic speaker. The ultrasonic speaker periodically broadcasts an auxiliary signal in the microphone array, which causes the attack signal of any frequency to undergo down-conversion under nonlinear action to generate a difference frequency signal with a frequency in the range of [10kHz, 20kHz]. The frequency range of [10kHz, 20kHz] is specifically chosen because the normal speaking voice is generally below 4kHz, thus avoiding the influence of environmental noise on the detection and not affecting the normal use of the device.

[0038] 2. The present invention proposes an ultrasonic attack detection and localization method based on a microphone array. This method utilizes the nonlinear effect of microphone hardware to detect potential ultrasonic attacks, extracts features from the difference frequency signal, calculates the time delay between each microphone pair, and uses an SVM classifier to locate the ultrasonic attack position. This technology can not only detect ultrasonic attack events in a timely and efficient manner, but also accurately measure the attacker's position in the first instance, thereby eliminating threats for users in a timely manner and taking timely countermeasures, thus playing a role in protecting information security and property security. Attached Figure Description

[0039] Figure 1 This is a flowchart of an ultrasonic attack detection and localization method based on a microphone array according to the present invention;

[0040] Figure 2 This is a schematic diagram illustrating the principle of converting auxiliary signals and attack signals into difference frequency signals for detection according to the present invention.

[0041] Figure 3 This is a schematic diagram of the detection auxiliary signal encoding of the present invention;

[0042] Figure 4 This is a schematic diagram illustrating the matching of the difference frequency signal and the auxiliary signal template used in this invention;

[0043] Figure 5 This is a first schematic diagram of the positioning ultrasonic attack source of the present invention;

[0044] Figure 6 This is a second schematic diagram of the positioning ultrasonic attack source of the present invention;

[0045] Figure 7 This is a simulation diagram of the microphone arrival difference convergence according to the present invention;

[0046] Figure 8 This is a diagram of the microphone arrival difference convergence experiment of the present invention. Detailed Implementation

[0047] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0048] Please see Figure 1 A method for detecting and locating ultrasonic attacks based on a microphone array includes the following steps:

[0049] S1: Set an ultrasonic speaker in the microphone array to periodically broadcast an auxiliary signal, which is down-converted to a possible ultrasonic attack signal, thereby monitoring for the presence of ultrasonic attacks.

[0050] The specific steps of step S1 include:

[0051] S101: An ultrasonic loudspeaker with multiple frequency broadcasts is placed in the microphone array to periodically broadcast an auxiliary signal. This signal undergoes a down-conversion with an attack signal of any frequency under nonlinear action, generating a difference frequency signal within the range of [10kHz, 20kHz], which can then be detected by us.

[0052] Sg = cos(2πf1t + 2πf2t + 2πf3t) (1),

[0053] Sg is the auxiliary signal, and f1, f2, and f3 are the three frequencies of the multi-tone signal, which are 25kHz, 45kHz, and 65kHz, respectively.

[0054] like Figure 2 As shown, due to the frequency band of the nonlinear effect and the hardware limitations of the ultrasonic loudspeaker, the attacker can choose an attack signal carrier frequency within [22kHz, 75kHz]. The auxiliary signal we use can ensure that any frequency attack signal chosen by the attacker will be down-converted with two adjacent auxiliary signals and detected by us.

[0055] S102: Encode the auxiliary signal to reduce environmental noise interference in subsequent signal analysis. Through experiments, we found that even within the preset low background noise frequency band [10kHz, 20kHz], our down-converted signal is still affected by some sound interference, such as music playback and unpacking sounds. We address this issue by performing simple OOK modulation on the auxiliary signal, such as... Figure 3As shown, an auxiliary signal with a duration of 500ms is divided into three parts: a preamble, a guard interval, and a continuous segment. The preamble is randomly assigned a special sequence of 0s and 1s to distinguish the signal from environmental noise when matching the down-converted signal with the auxiliary signal. The guard interval is to eliminate echo interference, and the continuous segment provides a sufficient length of audio samples for the subsequent positioning module.

[0056] S2: Preprocess the acquired data by filtering, denoising, segmenting, and standardizing the signals received by the microphone;

[0057] The specific steps of step S2 include:

[0058] S201: First, use a bandpass filter with a cutoff frequency of 10-20kHz to remove frequency band interference outside the difference frequency signal. Then, use a moving average filter to remove outliers and glitches, i.e., remove high-frequency noise and smooth the signal. Perform noise reduction processing on the multi-channel data collected by the microphone array. Set the cutoff frequency to [10kHz, 20kHz] to reduce the influence of environmental noise concentrated in the low frequency.

[0059] S202: Standardize the data, mainly including segmentation and Z-Score normalization. The segmentation operation provides audio data of the same size for subsequent steps, while Z-Score normalization eliminates errors caused by hardware differences between different microphones.

[0060] S3: Detect the preprocessed signal, perform spectrum peak detection on the difference frequency signal, find outliers, and then perform template matching with the auxiliary signal;

[0061] The specific steps of step S3 include:

[0062] S301: Detect whether there is a generated difference frequency signal in the audio signal. If there is a difference frequency signal, determine the location of the difference frequency signal and proceed to step S302. If no difference frequency signal is detected, the system outputs "no attack".

[0063] The specific implementation process of step S301 is as follows:

[0064] The attack signal is set as SA=(m(t)+ɑ)cos(2fat)(2),

[0065] Where (m(t)+ɑ) is the illegal voice command modulated by the attacker, and fa is the carrier signal frequency used by the attacker;

[0066] The attack signal and the auxiliary signal broadcast by the ultrasonic loudspeaker undergo down-conversion. Substituting this into formula (1), the following signal is generated: SM=(m(t)+α)cos(2π(fa-fg)t)(3),

[0067] First, perform FFT transformation on the down-converted signal, that is, process formula (3), and perform peak detection within the preset frequency band to determine whether there is an ultrasonic attack signal. Since the attack signal will be down-converted to the auxiliary signal of any two adjacent frequencies set by us and fall into the frequency band of [10kHz, 20kHz], we only need to pay attention to the abnormal peak value in this frequency range to determine whether there is an ultrasonic attack signal. If there is no ultrasonic attack signal, the system outputs no attack; if there is an ultrasonic attack signal, proceed to step S302.

[0068] S302: Perform spectral analysis on the difference frequency signal to identify outliers. Use a cross-correlation algorithm to match the difference frequency signal with the auxiliary signal to determine if the abnormal peak in S301 is caused by the down-conversion of the ultrasonic attack signal. This will help determine if it is indeed an ultrasonic attack signal. Figure 4 As shown, a cross-correlation algorithm is used to match the difference frequency signal with the auxiliary signal used to determine that the abnormal peak in S301 is caused by the ultrasonic attack signal and its down-conversion. When the real attack signal arrives, the correlation will have a peak. The height of this correlation peak is not sensitive to changes in the signal-to-noise ratio. In addition, spike noise is unlikely to have this specific coded data pattern, so its correlation value is very low. This can determine the occurrence of the ultrasonic event.

[0069] S4: Locate the ultrasonic attack source, extract the time delay between each microphone pair as a feature, and input it into the SVM classifier for localization.

[0070] The specific steps of step S4 include:

[0071] S401: Locates the ultrasonic attack source, extracts features from the difference frequency signal, performs cross-correlation calculations on each microphone pair, and obtains the time delay difference (TDOA) between the signals;

[0072] Taking a six-microphone array as an example, data from the six microphones is acquired through multi-channel acquisition. This method performs cross-correlation calculation on each microphone pair to obtain the time delay difference (TDOA) between signals, thus obtaining C62 = 15 pairs of time delay differences. These are then used as features and input into a machine learning classifier, such as... Figure 5 and Figure 6 As shown, the TDOA value is different when the attacker is located in different directions of the microphone array, which can be used as a basis for locating the attack source.

[0073] S402: Centered on the microphone array with a radius of 0.5m, it collects data from different locations of the attack source and establishes a database. The data for different locations includes the specific values ​​of the horizontal and vertical angles of the attack source. Due to the extremely small spacing between the array elements, when the distance between the attack source and the microphone array exceeds a certain distance, the arrival delay difference of the microphone pairs will converge. Figure 7 and Figure 8 As shown, we have demonstrated this conclusion using both simulation and experimental methods, so it is not necessary to collect data for all distances.

[0074] S403: Use an SVM classifier for localization and feed the results back to the system.

[0075] S5: Output the result. The system indicates whether there is an ultrasonic attack. If an attack is detected, it will continue to output the ultrasonic attack location result consisting of the horizontal and vertical angles. If there is no ultrasonic attack, it will output the acquired noise-reduced frequency.

[0076] In summary, this invention proposes a microphone array-based ultrasonic attack detection and localization method. It utilizes a microphone array, widely used in smart devices, and an ultrasonic speaker. The ultrasonic speaker periodically broadcasts an auxiliary signal within the microphone array, causing a down-conversion between the signal and any frequency of attack signal under nonlinear effects, generating a difference frequency signal within the range of [10kHz, 20kHz]. This difference frequency signal is then detected. The nonlinear effect of the microphone hardware is used to detect potential ultrasonic attacks. Features are extracted from the difference frequency signal, the time delay between each microphone pair is calculated, and an SVM classifier is used to locate the ultrasonic attack position. This technology not only detects ultrasonic attack events promptly and efficiently but also accurately measures the attacker's location in the first instance, thereby eliminating threats for users and enabling timely countermeasures to protect information and property security.

[0077] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for detecting and locating ultrasonic attacks based on a microphone array, characterized in that, Includes the following steps: S1: An ultrasonic loudspeaker is placed in the microphone array to periodically broadcast an auxiliary signal. This signal is down-converted to detect any ultrasonic attacks. The auxiliary signal is encoded to reduce environmental noise interference in subsequent signal analysis. This is achieved by simple OOK modulation of the auxiliary signal. A 500ms auxiliary signal is divided into three parts: a preamble, a guard interval, and a continuous segment. The preamble is randomly assigned a special sequence of 0s and 1s to distinguish the signal from environmental noise when matching the down-converted signal with the auxiliary signal. The guard interval is used to eliminate echo interference, and the continuous segment provides a sufficient length of audio samples for the subsequent positioning module. S2: Preprocess the acquired data by filtering, denoising, segmenting, and standardizing the signals received by the microphone; S3: Detect the preprocessed signal, perform spectrum peak detection on the difference frequency signal, find outliers, and then perform template matching with the auxiliary signal; S4: Locate the ultrasonic attack source, extract the time delay between each microphone pair as a feature, and input it into the SVM classifier for localization. S5: Output results. The system indicates whether an ultrasonic attack exists and the location of the ultrasonic attack source.

2. The ultrasonic attack detection and localization method based on a microphone array as described in claim 1, characterized in that: The specific steps of step S1 include: S101: An ultrasonic loudspeaker with multiple frequency broadcasts is placed in the microphone array to periodically broadcast an auxiliary signal. This signal is then down-converted with an attack signal of any frequency under nonlinear action, generating a difference frequency signal within the range of [10kHz, 20kHz], which can then be detected. Sg = cos(2πf1t + 2πf2t + 2πf3t) (1), Sg is the auxiliary signal, and f1, f2, and f3 are the three frequencies of the polyphonic signal, which are 25kHz, 45kHz, and 65kHz respectively. S102: Encode the auxiliary signal to reduce the interference of environmental noise in subsequent signal analysis.

3. The ultrasonic attack detection and localization method based on a microphone array as described in claim 2, characterized in that: The specific steps of step S2 include: S201: Performs noise reduction processing on the multi-channel data acquired by the microphone array, mainly including low-pass filtering and smoothing filtering; S202: Standardize the data, mainly including segmentation and Z-Score standardization.

4. The ultrasonic attack detection and localization method based on a microphone array as described in claim 3, characterized in that: The specific operation process of step S201 is as follows: Use a bandpass filter with a cutoff frequency of 10-20 kHz to remove frequency band interference other than the difference frequency signal; Using a moving average filter, outliers and glitches are removed, i.e., high-frequency noise is removed to smooth the signal.

5. The ultrasonic attack detection and localization method based on a microphone array as described in claim 4, characterized in that: The specific steps of step S3 include: S301: Detect whether there is a generated difference frequency signal in the audio signal. If there is a difference frequency signal, determine the location of the difference frequency signal and proceed to step S302. If no difference frequency signal is detected, the system outputs "no attack". S302: Perform spectrum value detection on the difference frequency signal, find outliers, and use a cross-correlation algorithm to match the difference frequency signal with the auxiliary signal to determine whether it is an ultrasonic attack signal.

6. The ultrasonic attack detection and localization method based on a microphone array as described in claim 5, characterized in that: The specific implementation process of step S301 is as follows: The attack signal is set as SA = (m(t) +ɑ )cos(2fat) (2), Where (m(t) +ɑ) is the illegal voice command modulated by the attacker, and fa is the carrier signal frequency used by the attacker; The attack signal and the auxiliary signal broadcast by the ultrasonic loudspeaker undergo downconversion. Substituting them into formula (1), the following signal is generated: SM = (m(t) + α)cos(2π(fa − fg)t) (3), Perform FFT transformation on the down-conversion signal, that is, process formula (3), perform peak detection within the preset frequency band, and determine whether there is an ultrasonic attack signal. If there is no ultrasonic attack signal, the system outputs no attack; if there is an ultrasonic attack signal, proceed to step S302.

7. The ultrasonic attack detection and localization method based on a microphone array as described in claim 1, characterized in that: The specific steps of step S4 include: S401: Locates ultrasonic attack signals, extracts features from difference frequency signals, performs cross-correlation calculations on each microphone pair, and obtains the time delay difference (TDOA) between signals; S402: Centered on the microphone array, with a radius of 0.5m, it collects data from different locations of the attack source and establishes a database; S403: Use an SVM classifier for localization and feed the results back to the system.

8. The ultrasonic attack detection and localization method based on a microphone array as described in claim 7, characterized in that: The data at different locations in step S402 includes the specific values ​​of the horizontal and vertical angles of the attack source.

9. The ultrasonic attack detection and localization method based on a microphone array as described in claim 8, characterized in that: The specific steps of step S5 include: the system indicating whether there is an ultrasonic attack; if an attack is detected, it continues to output the ultrasonic attack location result composed of horizontal and vertical angles; if there is no ultrasonic attack, it will output "no attack".

Images