Ultrasonic wave event spoofing attack defense method and device

Abstract

The invention relates to the technical field of information security, in particular to an ultrasonic event spoofing attack defense method and device, and the method comprises the steps: carrying out the preprocessing and ultrasonic attack detection of an environment audio based on a signal time domain feature and a frequency spectrum roll-off point, and judging whether the recording process is subjected to an ultrasonic attack or not; if the sound event cluster is subjected to the ultrasonic attack, performing event spoofing detection on each sound event cluster based on a signal spectrum false index to judge whether false events injected by an attacker are mixed in the cluster or not; and if false events are mixed, performing true and false event separation on each sound event cluster in which the false events are mixed so as to obtain a true sound event cluster after the false events are eliminated, and defending an ultrasonic wave event spoofing attack. According to the invention, whether the recording process suffers from the ultrasonic attack and whether false events are mixed into the sound event cluster can be detected, and the mixed false events can be eliminated to obtain the real sound event cluster, so that effective defense against the ultrasonic event spoofing attack is realized.

Description

Technical Field

[0001] The present application relates to the field of information security technology, and in particular to a method and device for defending against ultrasonic event deception attacks. Background Art

[0002] In related technologies, with the rapid development of home IoT technology, smart home systems integrate various device sensors and intelligent control systems to achieve automated control and intelligent interaction of home environments, bringing users a comfortable, convenient, safe and efficient smart home life experience. In recent years, in order to fully meet the flexible and changeable personalized life needs of users, smart home systems have been deeply integrated with artificial intelligence technology. By analyzing and processing home environment data collected by device sensors, learning and understanding users' daily behavior patterns, autonomous intelligent decision-making can be achieved, providing users with more accurate and detailed customized intelligent services. For example, the system can use microphones and other sound sensors to record the user's home environment audio within a day, detect all sound events that occur during the period, and cluster and divide them according to the differences in signal characteristics of different events to form multiple event clusters, and then analyze the frequency and time distribution of various events, and preliminarily understand the user's living habits and activity patterns, providing a basis for intelligent decision-making.

[0003] However, in the related art, when the smart home system uses environmental audio data to learn user behavior habits, it may be interfered by ultrasonic event spoofing attacks. Attackers can take advantage of the nonlinear hardware defects of the microphone to modulate pre-recorded event sounds on the ultrasonic carrier, generate attack signals that are inaudible to the human ear, and inject them into the device microphone in the recording. When the microphone receives these ultrasonic attack signals, it will demodulate and restore the original sound event signals, thereby forging the occurrence of sound events without attracting the user's attention. And when clustering sound events, smart home systems usually focus on the waveform differences of different event signals in the time domain. Therefore, attackers can carefully design the injected false sound events to have signal waveforms similar to those of real events in the user's home environment (such as the sounds of daily activities such as knocking on the door and closing the door), causing the system to mistakenly confuse these false events with real sound events of the same type and classify them all into the same event cluster. A large number of false events mixed in the cluster will mislead the system's analysis of the frequency and time distribution of events, causing serious deviations in the system's learning and understanding of users' daily behavior patterns, making it easy to make erroneous intelligent decisions. This will not only significantly reduce the user experience, but may also bring security risks and affect users' normal lives, which needs to be solved urgently. Summary of the invention

[0004] The present application provides a method and device for defending against ultrasonic event deception attacks to solve the problem in the related art that a smart home system is susceptible to interference from ultrasonic event deception attacks when using environmental audio data to learn user behavior habits, and is easily confused with false sound events secretly injected by attackers with real events, resulting in deviations in the learning and understanding of users' daily behavior patterns, thereby making erroneous intelligent decisions, which will not only significantly reduce the user experience, but may also bring security risks and affect users' normal lives.

[0005] The first aspect of the present application provides a method for defending against ultrasonic event deception attacks, comprising the following steps: pre-processing ambient audio based on signal time domain characteristics to obtain multiple sound event clusters consisting of a series of event signal blocks that meet preset waveform similarity conditions; performing ultrasonic attack detection on the ambient audio based on a signal spectrum roll-off point to determine whether the ambient audio has been subjected to an ultrasonic attack during the recording of the ambient audio; if subjected to the ultrasonic attack, performing event deception detection on each of the multiple sound event clusters based on a signal spectrum false index to determine whether each sound event cluster is mixed with false events injected by an attacker; if mixed with false events injected by the attacker, performing true and false event separation on each sound event cluster detected to be mixed with false events based on signal frequency domain characteristics to obtain a real sound event cluster after removing the false events, thereby defending against ultrasonic event deception attacks.

[0006] Optionally, in one embodiment of the present application, the ambient audio is preprocessed based on the signal time domain characteristics to obtain multiple sound event clusters consisting of a series of event signal blocks that meet preset waveform similarity conditions, including: taking the absolute value of the ambient audio data and performing filtering and smoothing processing to obtain a filtered audio signal; performing frame processing on the filtered audio signal to obtain an audio signal frame sequence of a target length, calculating the short-time energy value of each frame signal in the audio signal frame sequence, and marking each frame signal as an event signal or background noise according to the short-time energy value to determine multiple event signal frames; merging the multiple event signal frames in the audio signal frame sequence whose adjacent time intervals are less than a preset threshold into continuous event signal blocks to obtain a sound event sequence; extracting time domain features according to the absolute value of the signal amplitude of each event signal block in the sound event sequence to obtain a first feature vector that meets a first preset condition, and generating a first feature vector set of all events; performing dimensionality reduction processing on the first feature vector set of all events and performing clustering to obtain the multiple sound event clusters.

[0007] Optionally, in one embodiment of the present application, the ultrasonic attack detection is performed on the ambient audio based on the signal spectrum roll-off point to determine whether the ambient audio is subjected to an ultrasonic attack during the recording of the ambient audio, including: extracting a first signal spectrum of the ambient audio; calculating a spectrum roll-off point within a target frequency range based on the first signal spectrum; if the spectrum roll-off point is higher than a preset detection threshold, it is determined that the audio is subjected to an ultrasonic attack.

[0008] Optionally, in one embodiment of the present application, if the ultrasonic attack occurs, event deception detection is performed on each of the multiple sound event clusters based on the signal spectrum false index to determine whether each sound event cluster is mixed with false events injected by the attacker, including: extracting the second signal spectrum of each event in each sound event cluster; calculating the false index of each event based on the second signal spectrum; and when the false index of any one of the events in each event is higher than a preset detection threshold, determining that the sound event cluster where the any one event is located is mixed with a false event injected by the attacker.

[0009] Optionally, in one embodiment of the present application, if a false event injected by the attacker is mixed in, then based on the signal frequency domain characteristics, true and false events are separated for each sound event cluster detected to be mixed with a false event, so as to obtain a real sound event cluster after eliminating the false events, to defend against ultrasonic event deception attacks, including: extracting frequency domain features based on the signal spectrum of each event signal block in the sound event cluster mixed with the false event, so as to obtain a second eigenvector that meets a second preset condition, and generating a second eigenvector set of all events; performing dimensionality reduction processing and clustering on the second eigenvector set of all events, so as to obtain the real sound event cluster after eliminating the false events.

[0010] The second aspect of the present application provides an ultrasonic event deception attack defense device, including: a processing module, which is used to pre-process the ambient audio based on the signal time domain characteristics to obtain a plurality of sound event clusters composed of a series of event signal blocks that meet the preset waveform similarity conditions; a detection module, which is used to perform ultrasonic attack detection on the ambient audio based on the signal spectrum roll-off point to determine whether the ambient audio is subjected to an ultrasonic attack during the recording process; a judgment module, which is used to perform event deception detection on each of the multiple sound event clusters based on the signal spectrum false index in the case of the ultrasonic attack to determine whether each sound event cluster is mixed with false events injected by the attacker; a defense module, which is used to separate true and false events for each sound event cluster detected to be mixed with false events based on the signal frequency domain characteristics in the case of the false events injected by the attacker, so as to obtain a real sound event cluster after eliminating the false events, so as to defend against ultrasonic event deception attacks.

[0011] Optionally, in one embodiment of the present application, the processing module includes: a processing unit, which is used to take the absolute value of the data of the ambient audio and perform filtering and smoothing processing to obtain a filtered audio signal; a marking unit, which is used to perform frame processing on the filtered audio signal to obtain an audio signal frame sequence of a target length, calculate the short-time energy value of each frame signal in the audio signal frame sequence, and mark each frame signal as an event signal or background noise according to the short-time energy value to determine a plurality of event signal frames; a merging unit, which is used to merge the plurality of event signal frames in the audio signal frame sequence whose adjacent time intervals are less than a preset threshold into continuous event signal blocks to obtain a sound event sequence; a first extraction unit, which is used to extract time domain features according to the absolute value of the signal amplitude of each event signal block in the sound event sequence to obtain a first feature vector that meets a first preset condition and generates a first feature vector set of all events; a first division unit, which is used to perform dimensionality reduction processing and cluster division on the first feature vector set of all events to obtain the plurality of sound event clusters.

[0012] Optionally, in one embodiment of the present application, the detection module includes: a second extraction unit, used to extract a first signal spectrum of the ambient audio; a first calculation unit, used to calculate a spectrum roll-off point within a target frequency range based on the first signal spectrum; and a first determination unit, used to determine that the ultrasonic attack has occurred when the spectrum roll-off point is higher than a preset detection threshold.

[0013] Optionally, in one embodiment of the present application, the judgment module includes: a third extraction unit, used to extract the second signal spectrum of each event in each sound event cluster; a second calculation unit, used to calculate the false index of each event based on the second signal spectrum; and a second judgment unit, used to determine, when the false index of any event in each event is higher than a preset detection threshold, that a false event injected by the attacker is mixed into the sound event cluster where any event is located.

[0014] Optionally, in one embodiment of the present application, the defense module includes: a fourth extraction unit, used to extract frequency domain features based on the signal spectrum of each event signal block in the sound event cluster mixed with false events, so as to obtain a second feature vector that meets a second preset condition, and generate a second feature vector set of all events; a second division unit, used to perform dimensionality reduction processing and cluster division on the second feature vector set of all events, so as to obtain the real sound event cluster after eliminating the false events.

[0015] The third aspect of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the ultrasonic event deception attack defense method as described in the above embodiment.

[0016] The fourth aspect of the present application provides a computer-readable storage medium, which stores a computer program. When the program is executed by a processor, it implements the above-mentioned ultrasonic event deception attack defense method.

[0017] The fifth aspect of the present application provides a computer program product, including a computer program, which, when executed, is used to implement the above-mentioned ultrasonic event deception attack defense method.

[0018] This application can detect whether the recording process of the ambient audio is subjected to ultrasonic attack and determine whether there are false events mixed in the sound event cluster, and can eliminate the mixed false events to obtain the real sound event cluster, thereby completing the defense against the ultrasonic event deception attack. Thus, the smart home system can detect and cluster the sound events of the ambient audio based on the time domain characteristics of the signal to obtain the sound event cluster, and use the signal spectrum difference between the normal audio and the victim audio to preliminarily judge whether the recording process of the ambient audio has been subjected to ultrasonic attack, and in the case of detecting the ultrasonic attack, further use the signal spectrum difference between the real sound event and the false event to identify and eliminate the false events mixed in the sound event cluster, and finally obtain the real sound event cluster without any false events, thereby improving the accuracy of the analysis of the frequency and time distribution of various events by the smart home system, ensuring that the system learning results can truthfully reflect the daily behavior patterns of users, and then provide a reliable basis for the system to make more accurate intelligent decisions, significantly improve the user's life experience, and ensure the user's life safety. This solves the problem in the related art that the smart home system is susceptible to interference from ultrasonic event deception attacks when using environmental audio data to learn user behavior habits, and is easily confused with false sound events secretly injected by attackers with real events, resulting in deviations in the learning and understanding of users' daily behavior patterns, thereby making wrong intelligent decisions, which will not only significantly reduce the user experience, but may also bring security risks and affect users' normal life.

[0019] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0021] Figure 1 A flowchart of an ultrasonic event deception attack defense method provided according to an embodiment of the present application;

[0022] Figure 2 This is a flow chart of an environmental audio preprocessing method according to an embodiment of the present application;

[0023] Figure 3 This is a flow chart of a method for defending against ultrasonic event deception attacks based on signal spectrum analysis according to an embodiment of the present application;

[0024] Figure 4 A schematic diagram of the structure of an ultrasonic event deception attack defense device provided according to an embodiment of the present application;

[0025] Figure 5 It is a schematic diagram of the structure of an electronic device provided according to an embodiment of the present application.

[0026] Reference numerals:

[0027] 10-Ultrasonic event deception attack defense device: 100-processing module, 200-detection module, 300-judgment module and 400-defense module; 501-memory, 502-processor and 503-communication interface. DETAILED DESCRIPTION

[0028] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0029] The following describes the ultrasonic event deception attack defense method and device of the embodiment of the present application with reference to the accompanying drawings. In view of the fact that the smart home system in the related art mentioned in the above background technology is susceptible to ultrasonic event deception attack interference when using environmental audio data to learn user behavior habits, it is easy to mistakenly confuse the false sound events secretly injected by the attacker with real events, resulting in deviations in the learning and understanding of the user's daily behavior patterns, thereby making wrong intelligent decisions, which will not only significantly reduce the user experience, but may also bring security risks and affect the user's normal life. The present application provides an ultrasonic event deception attack defense method, in which it can detect whether the recording process of the environmental audio is subjected to ultrasonic attack and whether there are false events mixed in the sound event cluster, and the mixed false events can be eliminated to obtain the real sound event cluster, thereby completing the defense against ultrasonic event deception attack. Therefore, the smart home system detects and clusters the sound events of the ambient audio based on the time domain characteristics of the signal to obtain the sound event clusters, and uses the signal spectrum difference between the normal audio and the victim audio to preliminarily judge whether the ambient audio has been attacked by ultrasonic waves during the recording process. In the case of detecting the ultrasonic attack, the signal spectrum difference between the real sound event and the false event is further used to identify and remove the false events mixed in the sound event cluster, and finally obtain the real sound event cluster without any false events, thereby improving the accuracy of the analysis of the frequency and time distribution of various events by the smart home system, ensuring that the system learning results can truly reflect the daily behavior patterns of users, and then provide a reliable basis for the system to make more accurate intelligent decisions, significantly improve the user's life experience, and ensure the user's life safety. Therefore, the problem that the smart home system in the related technology is easily disturbed by the ultrasonic event deception attack when using the ambient audio data to learn the user's behavior habits, and it is easy to mistakenly confuse the false sound events secretly injected by the attacker with the real events, resulting in deviations in the learning and understanding of the user's daily behavior pattern, thereby making wrong intelligent decisions, which will not only significantly reduce the user experience, but may also bring security risks and affect the user's normal life.

[0030] Specifically, Figure 1 A flowchart of an ultrasonic event deception attack defense method provided in an embodiment of the present application.

[0031] like Figure 1 As shown, the ultrasonic event deception attack defense method includes the following steps:

[0032] In step S101, the ambient audio is preprocessed based on the signal time domain characteristics to obtain a plurality of sound event clusters consisting of a series of event signal blocks that meet preset waveform similarity conditions.

[0033] In some embodiments, when the smart home system detects sound events in the clustered environment audio, it usually focuses on the waveform differences of different event signals in the time domain. Therefore, the attacker can carefully design the injected false sound events to have signal waveforms similar to those of real events in the user's home environment (such as the sounds of daily activities such as knocking on the door and closing the door), causing the system to mistakenly confuse these false events with real sound events of the same type and classify them all into the same event cluster. A large number of false events mixed into the cluster will mislead the system's analysis of the frequency and time distribution of event occurrences, causing serious deviations in the system's learning and understanding of the user's daily behavior patterns, making it easy to make wrong intelligent decisions, which will not only significantly reduce the user experience, but may even bring security risks and affect the user's normal life.

[0034] Therefore, the embodiment of the present application can first pre-process the collected ambient audio data based on the time domain characteristics of the event signal to obtain multiple sound event clusters composed of a series of event signal blocks that meet certain waveform similarity conditions, and then further process and judge these sound event clusters.

[0035] Next, this process is further elaborated.

[0036] Optionally, in one embodiment of the present application, the ambient audio is preprocessed based on the signal time domain characteristics to obtain multiple sound event clusters consisting of a series of event signal blocks that meet preset waveform similarity conditions, including: taking the absolute value of the ambient audio data and performing filtering and smoothing processing to obtain a filtered audio signal; performing frame processing on the filtered audio signal to obtain an audio signal frame sequence of a target length, calculating the short-time energy value of each frame signal in the audio signal frame sequence, and marking each frame signal as an event signal or background noise according to the short-time energy value to determine multiple event signal frames; merging multiple event signal frames in the audio signal frame sequence whose adjacent time intervals are less than a preset threshold into continuous event signal blocks to obtain a sound event sequence; extracting time domain features based on the absolute value of the signal amplitude of each event signal block in the sound event sequence to obtain a first eigenvector that meets a first preset condition, and generating a first eigenvector set of all events; performing dimensionality reduction processing on the first eigenvector set of all events and performing clustering to obtain multiple sound event clusters.

[0037] In the actual implementation process, when the embodiment of the present application pre-processes the ambient audio data based on the time domain characteristics of the event signal to obtain multiple sound event clusters composed of a series of event signal blocks that meet certain waveform similarity conditions, it can be divided into but not limited to five steps. Figure 2 This is a flow chart of an environmental audio preprocessing method according to an embodiment of the present application. Figure 2 As shown:

[0038] Step S201: Take the absolute value of the ambient audio data and perform filtering and smoothing processing to obtain a filtered audio signal. For example, the absolute value of the original audio data S collected by the device microphone can be taken, and the signal can be smoothed using, but not limited to, an exponentially weighted moving average (EWMA) filter to eliminate noise interference.

[0039] Specifically, in the embodiment of the present application, when performing EWMA filtering on the original audio data S, the EWMA value calculation method of each sample data point in the signal can be expressed as follows:

[0040] S EWMA (i) = α*S(i) + (1-α)*S EWMA (i-1)

[0041] Wherein, S(i) represents the amplitude of the i-th sample data point in the signal, α represents the smoothing factor, α can usually be but not limited to 0.1, and can be set or adjusted by professional and technical personnel in this field according to actual needs in specific applications.

[0042] Step S202: Frame the filtered audio signal to obtain an audio signal frame sequence of target length, calculate the short-time energy value of each frame signal in the audio signal frame sequence, and mark each frame signal as an event signal or background noise according to the short-time energy value to determine multiple event signal frames. EWMA Perform frame processing and obtain a length of m s The audio signal frame sequence And calculate each frame signal s i The short time energy (STE) of e Frames with STE values ​​lower than thr are identified as event signals. e The frames are judged as background noise.

[0043] Specifically, in the embodiment of the present application, the filtered audio signal S EWMA When performing frame processing, the frame length can usually be set to 20ms and the frame shift can be set to 10ms, but is not limited to this. In specific applications, the professional and technical personnel in this field can set or adjust them according to actual needs. i When calculating the short-time energy (STE), it is equal to the sum of the squares of the signal amplitudes of all sample points in the frame. The formula can be expressed as follows:

[0044]

[0045] Among them, s i(j) represents the signal amplitude of the jth sample point in the i-th frame signal, Indicates the number of sample points of the i-th frame signal. When performing event signal detection, the detection threshold thr can usually be, but is not limited to, e Set to the maximum STE value of the first 100 frames of signal, which can be expressed as follows:

[0046] thr e =max(STE(s1),STE(s2),…,STE(s 100 ))

[0047] Furthermore, in specific applications, it can also be set or adjusted by professional and technical personnel in this field according to actual needs.

[0048] Step S203: merge multiple event signal frames whose adjacent time intervals in the audio signal frame sequence are less than a preset threshold into a continuous event signal block to obtain a sound event sequence. m The event signal frames can be merged into continuous event signal blocks to obtain a sound event sequence E = {e1, e2, …, e n}.

[0049] Specifically, when merging event signal frames with small intervals, the embodiment of the present application can generally, but is not limited to, set the interval merging threshold thr m It is set to 0.5s, and can be set or adjusted by professionals in this field according to actual needs in specific applications.

[0050] Step S204: extracting time domain features according to the absolute value of the signal amplitude of each event signal block in the sound event sequence to obtain a first feature vector that meets the first preset condition, and generating a first feature vector set for all events. i , the absolute value of the signal amplitude can be used to extract the time domain features and obtain an n T The feature vector of Among them, the features include but are not limited to 10 features that intuitively describe the statistical characteristics of the signal amplitude, such as length, area under the curve, maximum value, mean value, variance, standard deviation, energy, mean square value, root mean square value and root amplitude, as well as 10 features that describe the signal waveform shape and energy time distribution characteristics, such as skewness, kurtosis, waveform factor, peak factor, pulse factor, margin factor, time domain centroid, time domain stretch, time domain skewness and time domain kurtosis.

[0051] Specifically, in the embodiment of the present application, for each event signal block e iWhen extracting time domain features, the length, area under the curve, maximum value, mean value, variance, standard deviation, energy, mean square value, root mean square value and root amplitude can be calculated by simple statistical methods. The specific definitions and calculation methods of skewness, kurtosis, waveform factor, peak factor, pulse factor, margin factor, time domain centroid, time domain stretch, time domain skewness and time domain kurtosis can be expressed as follows:

[0052] Skewness can be used to describe the degree of asymmetry of a signal waveform, which is equal to the ratio of the third-order center distance of the signal amplitude to the cube of the standard deviation. The formula can be expressed as follows:

[0053]

[0054] Among them, s(i) represents the amplitude of the i-th sample data point in the signal, N represents the number of sample points of the signal, Mean represents the mean, and Std represents the standard deviation.

[0055] Kurtosis can be used to describe the steepness of a signal waveform, which is equal to the ratio of the fourth-order center distance of the signal amplitude to the fourth power of the standard deviation. The formula can be expressed as follows:

[0056]

[0057] Waveform Factor can be used to describe the waveform shape of a signal, which is equal to the ratio of the root mean square value to the absolute average value. The formula can be expressed as follows:

[0058]

[0059] Among them, RMS stands for the root mean square value, which is equal to the arithmetic square root of the square mean of the signal amplitude. The formula can be expressed as follows:

[0060]

[0061] Crest Factor can be used to describe the extreme degree of the signal peak in the waveform. It is often used to detect the impact component in the signal. It is equal to the ratio of the signal peak value to the root mean square value. The formula can be expressed as follows:

[0062]

[0063] Here, Max represents the maximum value (i.e., peak value).

[0064] Impulse Factor is often used to detect the pulse component in the signal. It is equal to the ratio of the signal peak value to the absolute average value. The formula can be expressed as follows:

[0065]

[0066] The margin factor is equal to the ratio of the signal peak value to the square root amplitude, and the formula can be expressed as follows:

[0067]

[0068] Among them, SMR represents the square root amplitude, which is equal to the square of the arithmetic square root mean of the signal amplitude. The formula can be expressed as follows:

[0069]

[0070] The temporal centroid can be used to represent the time point where the signal energy is concentrated. It is equal to the weighted average of the sampling time of all sample points in the signal with the signal amplitude as the weight. The formula can be expressed as follows:

[0071]

[0072] Wherein, t(i) represents the sampling time of the i-th sample data point in the signal, and t(1)=0 is specified.

[0073] Temporal Spread can be used to describe the degree of discreteness of signal energy distribution relative to the time center of gravity. The formula can be expressed as follows:

[0074]

[0075] Temporal skewness can be used to describe the asymmetry of the signal energy distribution relative to the time center of gravity. The formula can be expressed as follows:

[0076]

[0077] Temporal Kurtosis can be used to describe the steepness of the signal energy distribution relative to the time center of gravity. The formula can be expressed as follows:

[0078]

[0079] Step S205: Perform dimensionality reduction processing and clustering on the first feature vector set of all events to obtain multiple sound event clusters. For example, the feature vector set of all events can be reduced by using, but not limited to, principal component analysis (PCA). Dimensionality reduction is performed, and all events can be clustered using, but not limited to, the K-means clustering algorithm to obtain k sound event clusters C = {c1, c2, ..., c k}. Among them, each event cluster c i Depend on event signal blocks with similar waveforms, that is

[0080] Specifically, when performing K-Means clustering, the embodiments of the present application can generally, but not limited to, use the Elbow Method or Silhouette Coefficient evaluation index to determine the optimal number of clusters k for clustering. In specific applications, professional and technical personnel in this field can select or adjust it according to actual needs.

[0081] In step S102, ultrasonic attack detection is performed on the ambient audio based on the signal spectrum roll-off point to determine whether the ambient audio is subjected to ultrasonic attack during the recording process.

[0082] In some embodiments, after preprocessing the ambient audio, the present application can detect whether the ambient audio has been subjected to an ultrasonic attack during recording. Specifically, the ambient audio can be subjected to an ultrasonic attack detection based on the spectral roll-off point of the ambient audio signal to preliminarily determine whether the device microphone has been subjected to an ultrasonic attack during recording the ambient audio.

[0083] The following is a further explanation of this process.

[0084] Optionally, in one embodiment of the present application, ultrasonic attack detection is performed on ambient audio based on a signal spectrum roll-off point to determine whether the ambient audio is subjected to an ultrasonic attack during the recording process of the ambient audio, including: extracting a first signal spectrum of the ambient audio; calculating a spectrum roll-off point within a target frequency range based on the first signal spectrum; if the spectrum roll-off point is higher than a preset detection threshold, determining that the audio is subjected to an ultrasonic attack.

[0085] During the actual implementation process, when the embodiment of the present application performs ultrasonic attack detection on the ambient audio based on the signal spectrum roll-off point to determine whether the ambient audio is subjected to an ultrasonic attack during the recording process, it can, but is not limited to, first extracting the first signal spectrum of the ambient audio, and then calculating the spectrum roll-off point within the target frequency range based on the first signal spectrum, and when the spectrum roll-off point is higher than a certain detection threshold, it is determined that the ambient audio has been subjected to an ultrasonic attack during the recording process.

[0086] For example, the embodiment of the present application can extract the signal spectrum of the entire audio data S and calculate its spectrum roll-off point within a specific frequency range. If the value is higher than a preset detection threshold thr u, it can be considered that the device microphone may have been attacked by ultrasonic waves during the recording process, and all k sound event clusters C are sent to the subsequent steps for further processing, otherwise all k sound event clusters C are directly used as the final output.

[0087] Specifically, when calculating the spectral roll-off point (Spectral Roll-off) for the entire audio signal, the embodiment of the present application can take the value between [F l ,F r ]The signal spectrum within the frequency range and find the frequency dividing point when the signal spectrum energy drops to τ% of the total energy. The formula can be expressed as follows:

[0088] satisfy

[0089] Among them, F l and F r Indicates the left and right boundaries of the signal spectrum frequency range, F l Usually, it can be but not limited to 10kHz, F r It is fixed at 24kHz (i.e., 1 / 2 of the microphone recording sampling rate), f(j) represents the jth frequency component in the signal spectrum, r(i) represents the signal amplitude of the i-th frequency component in the signal spectrum, and N f Indicates the number of frequency components in the signal spectrum. τ% can usually be, but not limited to, 85% to 95%. When performing ultrasonic attack detection, the detection threshold thr can usually be, but not limited to, u Set to 20kHz. In specific applications, F l , τ and thr u All of them can be set or adjusted by professionals in this field according to actual needs.

[0090] It should be noted that the specific formula parameter values ​​and detection thresholds can be set or adjusted by professional and technical personnel in this field according to actual conditions and needs. The embodiments of this application are only for illustrative purposes and are not specifically limited.

[0091] In step S103, if an ultrasonic attack occurs, event deception detection is performed on each of the multiple sound event clusters based on the signal spectrum false index to determine whether each sound event cluster is mixed with false events injected by the attacker.

[0092] Based on the relevant descriptions of other embodiments, it can be understood that if it is detected that the recording process of the ambient audio is subjected to an ultrasonic attack, the present application can further process all sound event clusters C. Specifically, event deception detection can be performed on each sound event cluster according to the spectral false index (Fake Index) of the event signal to determine whether each sound event cluster is mixed with false events injected by the attacker.

[0093] The following is a further explanation of this process.

[0094] Optionally, in one embodiment of the present application, if an ultrasonic attack occurs, event deception detection is performed on each sound event cluster of multiple sound event clusters based on the signal spectrum false index to determine whether each sound event cluster is mixed with false events injected by the attacker, including: extracting the second signal spectrum of each event in each sound event cluster; calculating the false index of each event based on the second signal spectrum; and when the false index of any event in each event is higher than a preset detection threshold, determining that the sound event cluster where any event is located is mixed with false events injected by the attacker.

[0095] During the actual implementation process, when the embodiment of the present application performs event deception detection on each sound event cluster based on the signal spectrum false index to determine whether a false event injected by an attacker is mixed into the cluster, it can, but is not limited to, first extracting the second signal spectrum of each event in the sound event cluster, and then calculating the false index of each event based on the second signal spectrum, and when there is an event whose false index is higher than a certain detection threshold, that is, when the false index of any event is higher than a certain detection threshold, it can be determined that a false event injected by an attacker is mixed into the sound event cluster.

[0096] For example, the embodiment of the present application can extract each sound event cluster c i Each event The signal spectrum is obtained by calculating its fake index. If the fake index of an event in the cluster is higher than the preset detection threshold thr f , it can be considered that the sound event cluster may be mixed with false sound events injected by the attacker using ultrasound. k} is divided into two categories: one is the sound event cluster C mixed with false events f , will be sent to the subsequent steps for further processing; the other is the real sound event cluster C that is not mixed with false events t , will be directly included as part of the final output.

[0097] Specifically, in the embodiment of the present application, for each event signal block When calculating the spectrum fake index, the spectrum fake index can be equal to the logarithm of the ratio of the area occupied by the high-frequency component to the low-frequency component in the signal spectrum. The formula can be expressed as follows:

[0098]

[0099] Among them, f1 can usually be but not limited to 12kHz (i.e. 1 / 4 of the microphone recording sampling rate), and f2 is fixed to 24kHz (i.e. 1 / 2 of the microphone recording sampling rate). When performing event deception detection, the detection threshold thr can usually be but not limited to f Set to -0.5. In specific applications, f1 and thr f All of them can be set or adjusted by professionals in this field according to actual needs.

[0100] It should be noted that the specific formula parameter values ​​and detection thresholds can be set or adjusted by professional and technical personnel in this field according to actual conditions and needs. The embodiments of this application are only for illustrative purposes and are not specifically limited.

[0101] In step S104, if a false event injected by an attacker is mixed in, true and false events are separated for each sound event cluster detected to be mixed with a false event based on the signal frequency domain characteristics, so as to obtain a real sound event cluster after eliminating the false event, thereby defending against ultrasonic event deception attacks.

[0102] As a possible implementation method, after detecting false events injected by an attacker mixed into a sound event cluster, the present application can separate true and false events for each sound event cluster detected to be mixed with false events based on the frequency domain characteristics of the event signal to eliminate all false events, thereby obtaining the true sound event cluster and achieving defense against ultrasonic event deception attacks.

[0103] The following is a further explanation of this process.

[0104] Optionally, in one embodiment of the present application, if false events injected by an attacker are mixed in, true and false events are separated for each sound event cluster detected to be mixed with false events based on signal frequency domain characteristics to obtain a real sound event cluster after eliminating the false events, thereby defending against ultrasonic event deception attacks, including: extracting frequency domain features based on the signal spectrum of each event signal block in the sound event cluster mixed with the false events to obtain a second eigenvector that meets a second preset condition, and generating a second eigenvector set of all events; performing dimensionality reduction processing on the second eigenvector set of all events and performing clustering to obtain a real sound event cluster after eliminating the false events.

[0105] In certain embodiments, in the process of eliminating false events to obtain real sound event clusters, the present application needs to utilize the signal spectrum differences between real sound events and false events. Therefore, the present application can, but is not limited to, first extract the frequency domain features of each event signal block in the sound event cluster mixed with false events, obtain the second eigenvector that meets certain conditions, and then perform dimensionality reduction and clustering on the set of second eigenvectors of all events, thereby obtaining real sound event clusters.

[0106] For example, in the embodiment of the present application, for a sound event cluster c mixed with a false event, f,i Each event signal block in The signal spectrum can be used to extract frequency domain features to obtain an n F The feature vector of Among them, the features include but are not limited to five features that describe the signal spectrum profile shape and energy distribution characteristics, namely, spectrum centroid, spectrum extension, spectrum skewness, spectrum kurtosis and spectrum roll-off point. Then, the feature vector set of all events in the cluster can be analyzed using the PCA (Principal Component Analysis) method, but is not limited to it. Perform dimensionality reduction processing, and use the K-Means clustering algorithm to cluster and separate all real events and false events in the cluster, and obtain the real sound event cluster after removing the false events. Finally, all the real sound event clusters excluding the false events are As another part of the final output.

[0107] Specifically, in the embodiment of the present application, for each event signal block When performing frequency domain feature extraction, the specific definitions and calculation methods of spectrum centroid, spectrum stretch, spectrum skewness, spectrum kurtosis, and spectrum roll-off point can be expressed as follows:

[0108] The spectral centroid can be used to represent the frequency point where the energy in the signal spectrum is concentrated. It is equal to the weighted average of all frequency components in the signal spectrum with the signal amplitude as the weight. The formula can be expressed as follows:

[0109]

[0110] Where f(i) represents the i-th frequency component in the signal spectrum, r(i) represents the signal amplitude of the frequency component, and N f Indicates the number of frequency components in the signal spectrum.

[0111] Spectral spread can be used to describe the degree of dispersion of the signal spectrum energy distribution relative to the spectrum center of gravity. The formula can be expressed as follows:

[0112]

[0113] Spectral skewness can be used to describe the degree of asymmetry of the signal spectrum shape relative to the spectrum center of gravity. The formula can be expressed as follows:

[0114]

[0115] Spectral Kurtosis can be used to describe the steepness of the signal spectrum shape relative to the center of gravity of the spectrum. The formula can be expressed as follows:

[0116]

[0117] The spectral roll-off point can be used to indicate the frequency demarcation point when the signal spectrum energy drops to a specific percentage of the total energy. The formula can be expressed as follows:

[0118] satisfy

[0119] Among them, F l and F r Indicates the left and right boundaries of the signal spectrum frequency range, F l Usually, it can be but not limited to 0Hz, F r It is fixed at 24kHz (i.e. 1 / 2 of the microphone recording sampling rate), and τ% can usually be, but not limited to, 85% to 95%. l and τ can be set or adjusted by professionals in this field according to actual needs.

[0120] It should be noted that the specific formula parameter values ​​can be set or adjusted by professional and technical personnel in this field according to actual conditions and needs. The embodiments of this application are only for illustrative purposes and are not specifically limited.

[0121] The present application is described in detail below with reference to a specific embodiment.

[0122] Figure 3 This is a flow chart of an ultrasonic event deception attack defense method based on signal spectrum analysis according to an embodiment of the present application. Figure 3 As shown:

[0123] Step S301, obtaining the ambient audio collected by the device microphone.

[0124] Step S302: preprocessing the collected ambient audio based on the signal time domain characteristics.

[0125] Step S303: preprocessing is completed, and a plurality of sound event clusters consisting of a series of event signal blocks with similar waveforms are obtained.

[0126] Step S304: perform ultrasonic attack detection based on the signal spectrum roll-off point on the entire ambient audio.

[0127] Step S305, determining whether the ambient audio is subjected to an ultrasonic attack during recording; if no ultrasonic attack is detected, directly outputting all sound event clusters as final results.

[0128] Step S306: If an ultrasonic wave attack is detected, event deception detection based on a signal spectrum false index is performed on each sound event cluster.

[0129] Step S307, determining whether each sound event cluster is mixed with false events injected by the attacker, if no false events are detected in the cluster, the sound event cluster is also directly output as the final result.

[0130] Step S308: If a false event is detected in the cluster, the sound event cluster is processed in the next step.

[0131] Step S309, separating true and false events based on the signal frequency domain features for each sound event cluster detected to be mixed with false events.

[0132] Step S310, obtaining a real sound event cluster from which false events are eliminated, and outputting it as a final result, thereby completing the defense against ultrasonic event spoofing attacks.

[0133] According to the ultrasonic event deception attack defense method proposed in the embodiment of the present application, it is possible to detect whether the recording process of the ambient audio is subjected to an ultrasonic attack and to determine whether there are false events mixed into the sound event cluster, and to eliminate the mixed false events to obtain the real sound event cluster, thereby completing the defense against the ultrasonic event deception attack. Thus, the smart home system is implemented to detect and cluster the sound events of the ambient audio based on the time domain characteristics of the signal to obtain the sound event cluster, and to use the signal spectrum difference between the normal audio and the victim audio to preliminarily determine whether the recording process of the ambient audio is subjected to an ultrasonic attack, and in the case of detecting the ultrasonic attack, further use the signal spectrum difference between the real sound event and the false event to identify and eliminate the false events mixed into the sound event cluster, and finally obtain the real sound event cluster without any false events, thereby improving the accuracy of the analysis of the frequency and time distribution of various events by the smart home system, ensuring that the system learning results can truly reflect the daily behavior patterns of users, and then provide a reliable basis for the system to make more accurate intelligent decisions, significantly improve the user's life experience, and ensure the user's life safety. This solves the problem in the related art that the smart home system is susceptible to interference from ultrasonic event deception attacks when using environmental audio data to learn user behavior habits, and is easily confused with false sound events secretly injected by attackers with real events, resulting in deviations in the learning and understanding of users' daily behavior patterns, thereby making wrong intelligent decisions, which will not only significantly reduce the user experience, but may also bring security risks and affect users' normal life.

[0134] Secondly, the ultrasonic event deception attack defense device proposed in accordance with the embodiment of the present application is described with reference to the accompanying drawings.

[0135] Figure 4 It is a structural diagram of the ultrasonic event deception attack defense device according to an embodiment of the present application.

[0136] like Figure 4 As shown, the ultrasonic event deception attack defense device 10 includes: a processing module 100, a detection module 200, a judgment module 300 and a defense module 400.

[0137] The processing module 100 is used to pre-process the ambient audio based on the signal time domain characteristics to obtain a plurality of sound event clusters consisting of a series of event signal blocks that meet preset waveform similarity conditions.

[0138] The detection module 200 is used to perform ultrasonic attack detection on the ambient audio based on the signal spectrum roll-off point to determine whether the ambient audio is subjected to ultrasonic attack during the recording process.

[0139] The judgment module 300 is used to perform event deception detection on each sound event cluster of multiple sound event clusters based on the signal spectrum false index in the case of ultrasonic attack, so as to determine whether each sound event cluster is mixed with false events injected by the attacker.

[0140] The defense module 400 is used to separate true and false events for each sound event cluster detected to be mixed with false events based on the signal frequency domain characteristics, in the event of false events injected by the attacker, so as to obtain the real sound event cluster after eliminating the false events, and defend against ultrasonic event deception attacks.

[0141] Optionally, in one embodiment of the present application, the processing module 100 includes: a processing unit, a marking unit, a merging unit, a first extracting unit and a first dividing unit.

[0142] The processing unit is used to take the absolute value of the ambient audio data and perform filtering and smoothing processing to obtain a filtered audio signal.

[0143] The marking unit is used to perform frame processing on the filtered audio signal to obtain an audio signal frame sequence of a target length, calculate the short-time energy value of each frame signal in the audio signal frame sequence, and mark each frame signal as an event signal or background noise according to the short-time energy value to determine multiple event signal frames.

[0144] The merging unit is used to merge multiple event signal frames whose adjacent time intervals in the audio signal frame sequence are less than a preset threshold into continuous event signal blocks to obtain a sound event sequence.

[0145] The first extraction unit is used to extract time domain features according to the absolute value of the signal amplitude of each event signal block in the sound event sequence to obtain a first feature vector that meets a first preset condition and generate a first feature vector set for all events.

[0146] The first division unit is used to perform dimensionality reduction processing and cluster division on the first feature vector set of all events to obtain multiple sound event clusters.

[0147] Optionally, in one embodiment of the present application, the detection module 200 includes: a second extraction unit, a first calculation unit and a first determination unit.

[0148] The second extraction unit is used to extract the first signal spectrum of the ambient audio.

[0149] The first calculation unit is used to calculate a spectrum roll-off point within a target frequency range according to the first signal spectrum.

[0150] The first determination unit is configured to determine that an ultrasonic wave attack has occurred when a spectrum roll-off point is higher than a preset detection threshold.

[0151] Optionally, in one embodiment of the present application, the judgment module 300 includes: a third extraction unit, a second calculation unit and a second judgment unit.

[0152] The third extraction unit is used to extract the second signal spectrum of each event in each sound event cluster.

[0153] The second calculation unit is used to calculate the false index of each event according to the second signal spectrum.

[0154] The second determination unit is used to determine that a false event injected by an attacker is mixed into the sound event cluster where any event is located when the false index of any event in each event is higher than a preset detection threshold.

[0155] Optionally, in one embodiment of the present application, the defense module 400 includes: a fourth extraction unit and a second division unit.

[0156] Among them, the fourth extraction unit is used to extract frequency domain features according to the signal spectrum of each event signal block in the sound event cluster mixed with the false event, so as to obtain a second feature vector that meets the second preset condition and generate a set of second feature vectors for all events.

[0157] The second division unit is used to perform dimensionality reduction processing and cluster division on the second feature vector set of all events to obtain real sound event clusters after eliminating false events.

[0158] It should be noted that the above explanation of the embodiment of the ultrasonic event deception attack defense method is also applicable to the ultrasonic event deception attack defense device of this embodiment, and will not be repeated here.

[0159] According to the ultrasonic event deception attack defense device proposed in the embodiment of the present application, it is possible to detect whether the recording process of the ambient audio is subjected to an ultrasonic attack and determine whether there are false events mixed in the sound event cluster, and to remove the mixed false events to obtain the real sound event cluster, thereby completing the defense against the ultrasonic event deception attack. Thus, the smart home system is implemented to detect and cluster the sound events of the ambient audio based on the time domain characteristics of the signal to obtain the sound event cluster, and to use the signal spectrum difference between the normal audio and the victim audio to preliminarily determine whether the recording process of the ambient audio is subjected to an ultrasonic attack, and in the case of detecting the ultrasonic attack, further use the signal spectrum difference between the real sound event and the false event to identify and remove the false events mixed in the sound event cluster, and finally obtain the real sound event cluster without any false events, thereby improving the accuracy of the analysis of the frequency and time distribution of various events by the smart home system, ensuring that the system learning results can truly reflect the daily behavior patterns of users, and then provide a reliable basis for the system to make more accurate intelligent decisions, significantly improve the user's life experience, and ensure the user's life safety. This solves the problem in the related art that the smart home system is susceptible to interference from ultrasonic event deception attacks when using environmental audio data to learn user behavior habits, and is easily confused with false sound events secretly injected by attackers with real events, resulting in deviations in the learning and understanding of users' daily behavior patterns, thereby making wrong intelligent decisions, which will not only significantly reduce the user experience, but may also bring security risks and affect users' normal life.

[0160] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include:

[0161] A memory 501 , a processor 502 , and a computer program stored in the memory 501 and executable on the processor 502 .

[0162] When the processor 502 executes the program, the ultrasonic event spoofing attack defense method provided in the above embodiment is implemented.

[0163] Furthermore, the electronic device further comprises:

[0164] The communication interface 503 is used for communication between the memory 501 and the processor 502 .

[0165] The memory 501 is used to store computer programs that can be executed on the processor 502 .

[0166] The memory 501 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0167] If the memory 501, the processor 502 and the communication interface 503 are implemented independently, the communication interface 503, the memory 501 and the processor 502 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0168] Optionally, in a specific implementation, if the memory 501, the processor 502 and the communication interface 503 are integrated on a chip, the memory 501, the processor 502 and the communication interface 503 can communicate with each other through an internal interface.

[0169] The processor 502 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0170] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned ultrasonic event deception attack defense method.

[0171] The embodiment of the present application also provides a computer program product, including a computer program, which can run computer instructions. When the computer instructions are executed by a processor, the ultrasonic event deception attack defense method provided in the embodiment of the present application is implemented.

[0172] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0173] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0174] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0175] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or N wirings (electronic devices), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways as necessary and then storing it in a computer memory.

[0176] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented by hardware, as in another embodiment, it can be implemented by any one or a combination of multiple of the following technologies known in the art: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0177] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0178] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0179] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A method for defending against ultrasonic event deception attacks, characterized in that: The following steps are involved: Based on the signal time domain characteristics, the environmental audio is preprocessed to obtain a plurality of sound event clusters consisting of a series of event signal blocks that meet the preset waveform similarity conditions; Based on the signal spectrum roll-off point, the environmental audio is subjected to ultrasonic attack detection to determine whether the environmental audio is subjected to ultrasonic attack during the recording process; If the ultrasonic wave attack occurs, then based on the signal spectrum false index, event deception detection is performed on each of the multiple sound event clusters to determine whether false events injected by the attacker are mixed into each of the sound event clusters; If false events injected by the attacker are mixed in, true and false events are separated for each sound event cluster detected to be mixed with false events based on the signal frequency domain characteristics, so as to obtain the real sound event cluster after eliminating the false events, thereby defending against ultrasonic event deception attacks.

2. The method according to claim 1, characterized in that The method pre-processes the ambient audio based on the signal time domain characteristics to obtain a plurality of sound event clusters consisting of a series of event signal blocks that meet preset waveform similarity conditions, including: Taking an absolute value of the ambient audio data and performing filtering and smoothing processing to obtain a filtered audio signal; Performing frame processing on the filtered audio signal to obtain an audio signal frame sequence of a target length, calculating a short-time energy value of each frame signal in the audio signal frame sequence, and marking each frame signal as an event signal or background noise according to the short-time energy value to determine a plurality of event signal frames; Merging the plurality of event signal frames whose adjacent time intervals in the audio signal frame sequence are less than a preset threshold into a continuous event signal block to obtain a sound event sequence; Extracting time domain features according to the absolute value of the signal amplitude of each event signal block in the sound event sequence to obtain a first feature vector that meets a first preset condition, and generating a set of first feature vectors of all events; The first feature vector set of all the events is subjected to dimensionality reduction processing and clustering division to obtain the multiple sound event clusters.

3. The method according to claim 1, characterized in that The ultrasonic attack detection is performed on the ambient audio based on the signal spectrum roll-off point to determine whether the ambient audio is subjected to an ultrasonic attack during recording, including: Extracting a first signal spectrum of the ambient audio; Calculating a spectrum roll-off point within a target frequency range according to the first signal spectrum; If the spectrum roll-off point is higher than a preset detection threshold, it is determined that the ultrasonic wave attack has occurred.

4. The method according to claim 1, characterized in that: If the ultrasonic wave attack occurs, then based on the signal spectrum false index, event deception detection is performed on each of the multiple sound event clusters to determine whether each sound event cluster is mixed with false events injected by the attacker, including: Extracting a second signal spectrum of each event in each sound event cluster; Calculate a false index of each event according to the second signal spectrum; When the false index of any one of the events is higher than a preset detection threshold, it is determined that a false event injected by the attacker is mixed into the sound event cluster where the any one of the events is located.

5. The method according to claim 1, characterized in that If a false event injected by the attacker is mixed in, then based on the signal frequency domain characteristics, each sound event cluster detected to be mixed with a false event is separated into true and false events to obtain a true sound event cluster after removing the false event, and defend against ultrasonic event deception attacks, including: Extract frequency domain features according to the signal spectrum of each event signal block in the sound event cluster mixed with the false event to obtain a second feature vector that meets a second preset condition, and generate a set of second feature vectors of all events; The second feature vector set of all the events is subjected to dimensionality reduction processing and clustering division to obtain the real sound event clusters after the false events are eliminated.

6. An ultrasonic event deception attack defense device, characterized in that: include: A processing module, used for preprocessing the ambient audio based on the signal time domain characteristics to obtain a plurality of sound event clusters consisting of a series of event signal blocks that meet preset waveform similarity conditions; A detection module, used to perform ultrasonic attack detection on the ambient audio based on a signal spectrum roll-off point, so as to determine whether the ambient audio is subjected to ultrasonic attack during recording; A judgment module, configured to, in the case of being attacked by the ultrasonic wave, perform event deception detection on each of the plurality of sound event clusters based on a signal spectrum false index, so as to judge whether a false event injected by an attacker is mixed into each of the sound event clusters; The defense module is used to separate true and false events for each sound event cluster detected to be mixed with false events based on the signal frequency domain characteristics, in the case of false events injected by the attacker, so as to obtain the real sound event cluster after eliminating the false events, and defend against ultrasonic event deception attacks.

7. The device according to claim 6, characterized in that The processing module comprises: A processing unit, used for taking an absolute value of the ambient audio data and performing filtering and smoothing processing to obtain a filtered audio signal; a marking unit, configured to perform frame processing on the filtered audio signal to obtain an audio signal frame sequence of a target length, calculate a short-time energy value of each frame signal in the audio signal frame sequence, and mark each frame signal as an event signal or background noise according to the short-time energy value to determine a plurality of event signal frames; a merging unit, configured to merge the plurality of event signal frames whose adjacent time intervals in the audio signal frame sequence are less than a preset threshold into a continuous event signal block to obtain a sound event sequence; An extraction unit, configured to extract time domain features according to the absolute value of the signal amplitude of each event signal block in the sound event sequence, so as to obtain a first feature vector satisfying a first preset condition, and generate a first feature vector set of all events; The division unit is used to perform dimension reduction processing and cluster division on the first feature vector set of all the events to obtain the multiple sound event clusters.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the ultrasonic event deception attack defense method as described in any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the ultrasonic event deception attack defense method as described in any one of claims 1 to 5.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed, it is used to implement the ultrasonic event spoofing attack defense method as described in any one of claims 1-5.

Images