Deep breath identity authentication method based on sonar perception

Abstract

The invention discloses a deep breath identity authentication method based on sonar perception, and the method comprises the following steps: a user holds intelligent equipment or places the intelligent equipment on a desktop to carry out one-time deep breath, and a built-in acoustic module of the intelligent equipment transmits a sonar signal in a specific modulation mode and frequency; and echo signals and breath sound signals of the chest and abdomen combined movement of the user are received. A multi-stream identity authentication model is trained by extracting a breathing sound signal, a thoracic-abdominal joint movement signal and a correlation sequence between the breathing sound signal and the thoracic-abdominal joint movement signal, so that user identity authentication is realized. In order to prevent advanced replay attacks, an activity detection mechanism is designed according to the characteristic that chest and abdomen combined movement signals are interfered by environment and equipment during secondary playing and in combination with vibration data collected by a built-in acceleration sensor of intelligent equipment. The method is simple and convenient to operate and low in cost, can be used as an effective supplement of a mainstream authentication method, and improves the safety and reliability of identity authentication.

Description

Technical Field

[0001] The present invention belongs to the technical field of identity authentication, and in particular relates to a deep breathing identity authentication method based on sonar perception. Background Art

[0002] With the rapid development of ubiquitous computing and mobile computing, smart devices have become the most important connection point between humans, networks, and physical spaces. At the same time, their portability and accessibility pose challenges to user authentication, which is seen as an important access mechanism to ensure the reliability and security of this connection point. On the other hand, the rich perception and computing resources in smart devices have also prompted many researchers to implement local authentication by identifying users' biometrics. In fact, some biometric authentication methods have been widely studied and applied on smart devices, such as fingerprints, irises, faces, and voices. However, the rich resources in smart devices are a double-edged sword for voice- and face-based authentication, because personal voice and facial data can be easily copied and leaked, which increases the possibility of replay attacks and simulation attacks. In addition, the latest artificial intelligence big models have brought new security risks by using facial features and voice recordings to synthesize false information for telecommunications fraud.

[0003] At present, researchers have tried to use breathing, which is difficult to detect and imitate, for identity authentication. Some methods rely solely on breathing sounds for user identity authentication. Although they achieve good authentication performance, they are not suitable for noisy environments and are vulnerable to advanced replay attacks and simulation attacks. In addition, most methods that use breathing sounds for user identity authentication are wearable or contact methods, which makes users uncomfortable. Some methods use dedicated equipment (such as millimeter wave radar and Wi-Fi) for contactless user identity authentication, which adds additional overhead. In addition, neither of these two methods considers the impact of personal physiological state on breathing patterns. Changes in physiological state may directly lead to user identity authentication failures. Summary of the invention

[0004] In view of the problems existing in the existing methods, the present invention provides a deep breathing identity authentication method based on sonar perception. This method verifies the user's identity in a non-contact manner, overcomes the defect that the existing breathing authentication method is difficult to resist advanced replay and simulation attacks, and effectively reduces the impact of personal physiological state and environmental noise on user identity authentication. It is specifically implemented through the following technical solutions: A deep breathing identity authentication method based on sonar perception comprises the following steps: S1. The user holds the smart device or places it on the desktop according to his / her preference. The built-in speaker of the smart device transmits sonar signals with a preset modulation mode and frequency. At the same time, the built-in microphone of the smart device receives the sonar echo signals and breathing sound signals of the user's chest and abdominal movements. S2, denoising the received signal obtained in step S1 by using a high-pass filter to obtain a chest-abdomen combined motion signal; S3, calculating the energy sequence of the chest-abdomen joint motion signal obtained in step S2, and training the model to determine the start and end time endpoints of deep breathing; S4, determining the activity range of deep breathing in the received signal obtained in step S1 based on the deep breathing endpoint obtained in step S3; S5, performing high-pass and band-pass filtering on the received signal that has been processed by the endpoint in step S4 to obtain a chest-abdomen combined motion signal and a corresponding breathing sound signal; S6. Training a liveness monitoring classifier based on the chest-abdomen joint motion signal and the vibration data of the smart device caused by hand gripping. The liveness monitoring classifier is used to identify whether the source of the deep breathing is a real person or a replay attacker. If the liveness monitoring classifier detects that the deep breathing is not from a real person, the next step will not be performed. S7, extracting features from the chest-abdomen combined motion signal and the respiratory sound signal, calculating the correlation between the chest-abdomen combined motion signal and the corresponding respiratory sound signal, and obtaining a correlation sequence; S8. Design a multi-stream identity authentication model to support the input of the three-factor features of the chest-abdomen joint motion signal, breathing sound signal and correlation sequence obtained in step S7 to authenticate the user.

[0005] Furthermore, the sonar signal emitted by the smart device in step S1 is in the form of a frequency modulated continuous wave with a frequency range of 20kHz-24kHz, wherein each ultrasonic chirp signal is 50ms.

[0006] Furthermore, in step S2, the cutoff frequency of the high-pass filter is 20 kHz.

[0007] Furthermore, the method for determining the start and end time points of deep breathing in step S3 is specifically: calculating the short-time energy and short-time average amplitude sequence of the signal, and then inputting them into a sequence-to-sequence model based on a gated recurrent unit to determine the activity range of deep breathing.

[0008] Furthermore, in step S5, the cut-off frequency of the high-pass filter is 20 kHz, and the cut-off frequency of the band-pass filter is set to 500-3000 Hz.

[0009] Furthermore, the method for detecting replay attacks in step S6 is specifically as follows: first, synchronously obtain the chest-abdomen joint motion signal and use the built-in acceleration sensor of the smart device to capture the smart device vibration data caused by hand micro-movement; then extract the standard deviation, shape factor and pulse factor characteristics of the chest-abdomen joint motion signal and the maximum amplitude, signal amplitude area and average absolute value of the smart device vibration data; finally, input these six-dimensional features into the support vector classification model for identifying replay attackers.

[0010] Furthermore, the method for calculating the correlation in step S7 is to measure the chest-abdomen joint motion signal and the corresponding breathing sound signal through mutual information.

[0011] The method of the present invention has significant advantages over the existing breathing authentication methods: by introducing sonar signals to perceive the user's chest and abdomen joint movement, it not only effectively reduces the interference of environmental noise on the authentication results, but also reduces the risk of simulated attacks. In addition, the present invention uses the chest and abdomen joint movement signals and the vibration data of the smart device caused by hand holding to train the live monitoring classifier to identify and defend against advanced replay attacks, further improving the robustness of the system. The present invention also fully considers the impact of personal physiological state on user identity authentication, thereby ensuring the reliability of the authentication process under different physiological conditions. The sonar signal frequency used in the present invention is within the inaudible range of the human ear, does not interfere with the user, and has good user-friendliness. More importantly, the method can be directly deployed on commercial smart devices without the need for additional hardware support, thereby greatly reducing the implementation cost and hardware burden, and has broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 is a flow chart of the present invention; Figure 2 This is the feature extraction and identity authentication network diagram of the present invention. DETAILED DESCRIPTION

[0013] The present invention is further described below in conjunction with the accompanying drawings in order to better understand the technical solution.

[0014] The present invention discloses a deep breathing identity authentication method based on sonar perception, which uses the built-in acoustic module of smart devices such as smart phones and tablets to transmit FMCW sonar signals, and captures the echo signal of the chest and abdomen joint movement caused by the user's deep breathing and the breathing sound signal to verify the user's identity. In order to enhance the security of the system, the present invention uses the chest and abdomen joint movement signal and the vibration data of the smart device caused by hand grip to train the live monitoring classifier to identify and defend against advanced replay attacks.

[0015] Specifically, Figure 1 As shown, the provided deep breathing identity authentication method based on sonar perception includes the following steps: S1. The user holds the smart device or places it on the desktop according to his or her preference. The built-in speaker of the smart device transmits sonar signals with a pre-set modulation method and frequency. At the same time, its embedded microphone receives the sonar echo signals and breathing sound signals of the user's chest and abdominal movements.

[0016] In this step, the sonar signal emitted by the smart device is in the form of a frequency modulated continuous wave with a frequency range of 20kHz-24kHz, where each chirp signal is 50ms. The time domain representation of the chirp signal is as follows:

[0017] in , N∈Z, A represents the amplitude of the transmitted signal, fc = (F l + F h ) / 2 represents the center frequency, B = F h -F l represents bandwidth, T s Indicates the scan cycle.

[0018] In order to mitigate the power leakage caused by frequency discontinuity, the present invention applies a cosine window to each chirp signal: , Where M is the chirp length and α is the cosine fraction, which is used to control the length of the transition region at the edge of the window function.

[0019] S2. De-noise the received signal obtained in step S1 by using a high-pass filter to obtain a chest-abdomen joint motion signal. The cutoff frequency of the high-pass filter is set to 20 kHz.

[0020] S3. Deep breathing activities are usually accompanied by an increase in signal energy and a change in signal amplitude. Therefore, in order to detect the start and end points of deep breathing, the short-time energy and short-time average amplitude sequence of the chest-abdomen joint motion signal obtained in step S2 are calculated. Next, a sequence-to-sequence classification model based on a gated recurrent unit is trained. Then the two energy sequences are input into the classification model to obtain an output sequence of 0 and 1. 0 indicates no deep breathing, and 1 indicates deep breathing.

[0021] S4. Determine the activity range of deep breathing in the received signal obtained in step S1 based on the output sequence obtained in step S3.

[0022] S5. Perform high-pass and band-pass filtering on the received signal processed by the endpoint in step S4 to obtain a chest-abdomen joint motion signal and a corresponding breathing sound signal. The cutoff frequency of the high-pass filter is set to 20kHz. Since there is interference in the signal from the direct path from the speaker to the microphone, the cutoff frequency of the band-pass filter is set to 500-3000Hz.

[0023] Since the filter mainly processes the frequency domain of the signal, filtering may introduce some extreme peaks or valleys in the time domain. In order to deal with these extreme values, the median absolute deviation algorithm is used to detect outliers. Then, the outlier is replaced by the last non-outlier value.

[0024] S6. Breathing-based authentication faces the threat of advanced replay attacks, that is, attackers may hack into user devices, steal original recordings containing breathing features, and replay these recordings on other devices to deceive the authentication system. In theory, when the stolen deep breathing audio re-enters the authentication system: first, an analog signal is generated through the audio digital-to-analog converter (DAC, in the case of smartphones) of the attacker's device; then, this analog signal is amplified by the audio amplifier and converted into a sound wave signal through the speaker, propagates through the air, and is finally sampled by the smart device equipped with the authentication system. However, during this transmission process, the replayed deep breathing audio will be subject to two types of interference: one is device-related interference (such as nonlinear distortion of the audio amplifier), and the other is environment-related interference (such as multipath interference), which causes the time domain signal to become flat. In addition, when using smartphones, users usually hold the device in their hands or place it on the table. In the handheld state, deep breathing will naturally cause small movements of the hand, which are difficult to detect with the naked eye and difficult to accurately replicate during the attack. However, these small movements will be reflected by the vibration of the phone and accurately captured by the built-in accelerometer of the smartphone.

[0025] Therefore, in order to enhance the system's ability to resist advanced replay attacks, a live monitoring classifier was trained based on the chest-abdomen joint motion signals and the vibration data of the smart device caused by hand holding (collected by the built-in accelerometer of the smart device).

[0026] In this step, the chest-abdomen joint motion signal is first obtained synchronously, and the vibration data of the smart device caused by hand micro-movement is captured by the built-in acceleration sensor of the smart device; then the standard deviation, shape factor and pulse factor characteristics of the chest-abdomen joint motion signal and the maximum amplitude, signal amplitude area and average absolute value of the smart device vibration data are extracted; based on these six-dimensional features, a support vector classification model is trained to identify the replay attacker. If it is detected that the deep breathing is not from a real person, the next step will not be performed.

[0027] S7. Extract 18-dimensional features from the chest-abdomen joint motion signal (respiration duration, inhalation time, exhalation time, inhalation-expiration interval, inhalation-expiration time ratio, respiratory average, inhalation average, exhalation average, inhalation-expiration mean ratio, respiratory root mean square, inhalation root mean square, exhalation root mean square, inhalation-expiration root mean square ratio, inhalation kurtosis, exhalation kurtosis, respiratory standard deviation, respiratory shape factor, and respiratory pulse factor). Extract the Bark spectrum of the respiratory sound signal, and calculate the spectral centroid, spectral entropy, spectral roll-off point, and spectral extension of the Bark spectrum. Calculate the mutual information value of the chest-abdomen joint motion signal and the corresponding respiratory sound signal root mean square envelope in each time frame. The frame length is set to 1200, 2400, and 4800, and the frame shift is set to 1200.

[0028] S8. Design a multi-stream identity authentication model to support the input of the three-factor features of chest-abdomen joint motion signal, breathing sound signal and correlation sequence obtained in S7 to authenticate the user identity. Figure 2 As shown in the figure, the network consists of two parts: feature extraction and identity authentication. In the feature extraction part, the chest-abdomen joint motion signal features pass through four fully connected layer networks; the feature sequence and mutual information sequence of the breathing sound signal pass through the combined network of convolutional neural network (CNN) and bidirectional LSTM. After the features of the three input streams are concatenated, they are further compressed through self-attention, CNN and fully connected layers to extract features related to identity authentication. Finally, the Softmax layer is used for classification and the user's identity ID is output.

[0029] The present invention may be embodied as a system, method and / or computer program product in practical applications. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

Claims

1. A deep breathing identity authentication method based on sonar perception, characterized in that: The method comprises the following steps: S1. The user holds the smart device or places it on the desktop according to his / her preference. The built-in speaker of the smart device transmits sonar signals with a preset modulation mode and frequency. At the same time, the built-in microphone of the smart device receives the sonar echo signals and breathing sound signals of the user's chest and abdominal movements. S2, denoising the received signal obtained in step S1 by using a high-pass filter to obtain a chest-abdomen combined motion signal; S3, calculating the energy sequence of the chest-abdomen joint motion signal obtained in step S2, and training the model to determine the start and end time endpoints of deep breathing; S4, determining the activity range of deep breathing in the received signal obtained in step S1 based on the deep breathing endpoint obtained in step S3; S5, filtering the received signal after endpoint processing in step S4 by high-pass filter and band-pass filter respectively to obtain a chest-abdomen joint motion signal and a corresponding breathing sound signal; S6. Training a liveness monitoring classifier based on the chest-abdomen joint motion signal and the vibration data of the smart device caused by hand gripping. The liveness monitoring classifier is used to identify whether the source of the deep breathing is a real person or a replay attacker. If the liveness monitoring classifier detects that the deep breathing is not from a real person, the next step will not be performed. S7, extracting features from the chest-abdomen combined motion signal and the respiratory sound signal, calculating the correlation between the chest-abdomen combined motion signal and the corresponding respiratory sound signal, and obtaining a correlation sequence; S8. Design a multi-stream identity authentication model to support the input of the three-factor features of the chest-abdomen joint motion signal, breathing sound signal and correlation sequence obtained in step S7 to authenticate the user.

2. A deep breathing identity authentication method based on sonar perception as claimed in claim 1, characterized in that: The sonar signal emitted by the smart device in step S1 is in the form of a frequency modulated continuous wave with a frequency range of 20kHz-24kHz, wherein each ultrasonic chirp signal is 50ms.

3. A deep breathing identity authentication method based on sonar perception as claimed in claim 1, characterized in that: The method for determining the start and end time points of deep breathing in step S3 is specifically: calculating the short-time energy and short-time average amplitude sequence of the signal, and then inputting them into a sequence-to-sequence model based on a gated recurrent unit to determine the activity range of deep breathing.

4. A deep breathing identity authentication method based on sonar perception as claimed in claim 1, characterized in that: In step S5, the cut-off frequency of the high-pass filter is 20 kHz, and the cut-off frequency of the band-pass filter is set to 500-3000 Hz.

5. A deep breathing identity authentication method based on sonar perception as claimed in claim 1, characterized in that: The method for detecting replay attacks in step S6 is specifically as follows: first, synchronously obtain the chest-abdomen joint motion signal and use the built-in acceleration sensor of the smart device to capture the smart device vibration data caused by hand micro-movement; then extract the standard deviation, shape factor and pulse factor characteristics of the chest-abdomen joint motion signal and the maximum amplitude, signal amplitude area and average absolute value of the smart device vibration data; finally, input these six-dimensional features into the support vector classification model for identifying replay attackers.

6. A deep breathing identity authentication method based on sonar perception as claimed in claim 1, characterized in that: The method for calculating the correlation in step S7 is to measure the chest-abdomen joint motion signal and the corresponding breathing sound signal through mutual information.

Images