A friction acoustic-based security gesture password authentication method, system and device

Abstract

The application discloses a safe gesture password authentication method, system and device based on friction sound, first, a user draws an unlocking pattern on a screen by using a finger, a built-in microphone of a mobile device captures friction sound, and friction sound information is acquired; then, the unlocking pattern drawn by the user is matched with an unlocking pattern stored in the system during registration; if the pattern is incorrect, a result that the user is an illegal user is directly fed back; otherwise, the recorded audio is subjected to next-step processing; then, friction sound preprocessing is carried out, clean and enhanced sound signals are acquired; friction time is detected, and the friction sound is cut; finally, fingerprint features are extracted, a confidence score is given for each piece of friction sound, and after weighted processing, the user with a score greater than a preset value belongs to a legal user. Since the friction sound is related to the fingerprint features of the user, it is difficult for an attacker to imitate, and therefore, the application has good security.

Description

Technical Field

[0001] This invention belongs to the field of mobile terminal security technology, and relates to a secure gesture password authentication method, system and device, specifically a secure gesture password authentication method, system and device based on friction sound. Background Technology

[0002] As mobile devices (such as smartphones and tablets) offer increasingly diverse services, numerous apps have emerged that involve property security and personal privacy, including social networking, shopping, and payment apps. Gesture passwords play a crucial role in user authentication on mobile devices due to their ease of memorization and use. However, traditional gesture password authentication, which only verifies the correctness of the entered pattern, is no longer sufficient to counter threats and attacks from malicious actors.

[0003] Users often have personal preferences when setting gesture passwords, making the chosen patterns less resistant to guessing attacks. Gesture password unlocking strategies based on fingerprint friction sounds can effectively resist brute-force attacks. Shoulder spying attacks are a more powerful direct attack; when a legitimate user enters their gesture password, an attacker can easily spy on it through direct observation, a mirror, or a hidden pinhole camera. Furthermore, when a user draws their unlock pattern, they reveal side-channel information, providing opportunities for various attack methods. For example, the smartphone's accelerometer data, oil stains or heat left on the screen when drawing the pattern, and involuntary eye movements during unlocking can all be exploited by malicious actors to crack the user's gesture password. These attack methods pose a significant threat to people's information security. Summary of the Invention

[0004] To address the aforementioned technical issues, this invention integrates the friction sound generated by a finger sliding on the screen into gesture password input, providing a secure gesture password authentication method, system, and device based on friction sound. This effectively avoids shoulder spying attacks and side-channel attacks, and minimally increases the complexity of using gesture password unlocking.

[0005] The technical solution adopted by the method of the present invention is: a secure gesture password authentication method based on friction sound, comprising the following steps:

[0006] Step 1: The user draws an unlock pattern on the screen with their finger. The mobile device's built-in microphone captures the friction sound and obtains the friction sound information.

[0007] Step 2: Match the unlock pattern drawn by the user with the unlock patterns stored in the system during registration; if the pattern is incorrect, directly report that the user is an unauthorized user; otherwise, proceed with the recorded audio to the next step.

[0008] Step 3: Friction sound preprocessing, using one or more of the following processing methods, including background noise elimination, target signal enhancement, and wavelet re-denoising, to obtain a clean and enhanced sound signal;

[0009] Step 4: Detect the friction time and cut off the friction sound;

[0010] Step 5: Extract fingerprint features and assign a confidence score to each friction sound. After weighted processing, users whose scores are greater than the preset value are considered legitimate users.

[0011] The technical solution adopted by the system of the present invention is: a secure gesture password authentication system based on friction sound, comprising the following modules:

[0012] Module 1 is used by users to draw an unlock pattern on the screen with their fingers. The mobile device's built-in microphone captures the friction sound and obtains the friction sound information.

[0013] Module 2 is used to match the unlock pattern drawn by the user with the unlock patterns stored in the system during registration; if the pattern is incorrect, it will directly report that the user is an unauthorized user; otherwise, it will process the recorded audio for the next step.

[0014] Module 3 is used for friction sound preprocessing. It employs one or more processing methods, including background noise elimination, target signal enhancement, and wavelet re-denoising, to obtain a clean and enhanced sound signal.

[0015] Module 4 is used to detect friction time and cut the friction sound;

[0016] Module 5 is used to extract fingerprint features and give a confidence score for each friction sound. After weighted processing, users whose scores are greater than the preset value are considered legitimate users.

[0017] The technical solution adopted by the device of the present invention is: a secure gesture password authentication device based on friction sound, comprising:

[0018] One or more processors;

[0019] A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the friction sound-based secure gesture password authentication method.

[0020] Even if an attacker cracks a user's gesture password, they still need to verify the fingerprint rubbing sound. The uniqueness of the rubbing sound depends on the texture features of the contact surface (i.e., the user's fingerprint), and everyone's fingerprint is unique. Therefore, integrating the rubbing sound generated by a finger sliding on the screen into gesture password input can effectively avoid shoulder spying attacks and side-channel attacks, and minimally increase the complexity of using gesture password unlocking.

[0021] Compared to existing technologies, the advantages of this invention are: it effectively resists shoulder spying attacks, and even in the event of a leaked gesture password, the probability of device data leakage can be reduced to below 5%. Furthermore, because the friction sound is related to the user's fingerprint, it is difficult for attackers to imitate, thus providing excellent security. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the method principle of an embodiment of the present invention. Detailed Implementation

[0023] To facilitate understanding and implementation of the present invention by those skilled in the art, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0024] This invention primarily focuses on the security of gesture passwords on mobile terminals. Considering the friction sound generated when a user inputs a gesture password, it proposes a secure gesture password authentication mechanism based on this friction sound. This method fully utilizes the different friction sounds produced by the user's fingers and the screen when inputting a gesture password to research and explore an implicit authentication method using the user's fingerprint. This friction sound-based secure gesture password authentication mechanism can effectively resist shoulder spying attacks, reducing the probability of device leakage to below 5% even if the gesture password is leaked. Furthermore, because the friction sound is related to the user's fingerprint, it is difficult for attackers to imitate, providing excellent security.

[0025] See Figure 1 The present invention provides a secure gesture password authentication method based on friction sound, comprising the following steps:

[0026] Step 1: The user draws an unlock pattern on the screen with their finger. The mobile device's built-in microphone captures the friction sound and obtains the friction sound information.

[0027] In this embodiment, when the user draws an unlock pattern on the screen with their finger, the mobile device's built-in microphone simultaneously captures the friction sound. Specifically, this embodiment requires the user-set pattern to pass through at least four different points to unlock, which is a common setting for gesture passwords on most commercial mobile phones currently on the market.

[0028] Step 2: Match the unlock pattern drawn by the user with the unlock patterns stored in the system during registration; if the pattern is incorrect, directly report that the user is an unauthorized user; otherwise, proceed with the recorded audio to the next step.

[0029] In this embodiment, the user inputs an unlock pattern, and the system matches the user-drawn pattern with unlock patterns stored in the system during registration. If the pattern is incorrect, the system directly reports that the user is an unauthorized user. Otherwise, the client transmits the recorded audio in real time for further processing. This stage can directly call the device's existing gesture password authentication module.

[0030] Step 3: Friction sound preprocessing, using one or more of the following processing methods, including background noise elimination, target signal enhancement, and wavelet re-denoising, to obtain a clean and enhanced sound signal;

[0031] Most of the energy of ambient noise in different environments (such as offices, supermarkets, and streets) is located in low-frequency signals (e.g., less than 5kHz). The frequency of friction noise typically does not exceed 22kHz. Therefore, this embodiment selects a Butterworth bandpass filter to acquire friction noise signals in the 5-22kHz range, and uses a third-order Butterworth filter to maintain the smooth amplitude-frequency characteristics of the passband. Through this noise reduction process, the characteristic information is not distorted.

[0032] In this embodiment, the target signal enhancement processing uses multi-band spectral subtraction (MBSS) to filter out colored noise at different frequencies that affect the spectral uniformity of the audio after background noise cancellation, thereby enhancing the speech spectrum. This embodiment divides the friction sound spectrum into four non-overlapping frequency bands and estimates the posterior signal-to-noise ratio and corresponding subtraction coefficients for each band. Because there are no fundamental changes between frames in each frequency band, the enhancement method in this embodiment effectively avoids distortion of the friction sound.

[0033] In this embodiment, wavelet re-denoising processing first uses the Maximum Overlap Discrete Wavelet Transform (MODWT) to decompose the signal. In this embodiment, the Daubechies 3 (db3) wavelet is selected, and the wavelet decomposition of the signal at level 6 is calculated. Then, threshold denoising is used for further processing. In this stage, the threshold is adjusted according to the noise estimation of different levels. A threshold is selected for each level from 1 to 6, and a soft threshold is used for the detail coefficients. Finally, the wavelet coefficients obtained after processing are reconstructed using the Inverse Maximum Overlap Discrete Wavelet Transform (IMODWT) to obtain the denoised audio signal.

[0034] The specific algorithm process for wavelet denoising is explained below:

[0035] (1) First, the signal is decomposed using the maximum overlap discrete wavelet transform (MODWT) method.

[0036] (2) Then, threshold denoising is used for further processing. In this stage, the threshold is adjusted based on noise estimation at different levels, and soft thresholding is applied to the detail coefficients.

[0037] (3) Finally, the processed wavelet coefficients are reconstructed using the inverse maximum overlap discrete wavelet transform (IMODWT) method to obtain the denoised audio signal. The audio signal processed in step 3 is almost free of noise interference.

[0038] Compared to other denoising methods, wavelet transform exhibits better denoising performance at low signal-to-noise ratios. The denoised signal has a higher recognition rate. Furthermore, wavelet denoising is particularly effective for signals with abrupt changes.

[0039] Step 4: Detect the friction time and cut off the friction sound;

[0040] This embodiment first obtains clean and enhanced audio, then uses a speech activity detection (VAD) algorithm based on a Hidden Markov Model (HMM) to determine the probability of friction sounds occurring at different times. Only high-probability segments (probability greater than 90%) are considered and treated as coarse friction events. However, the resulting friction events will still include some silent segments and misjudged segments (such as the sticky sound produced when a finger leaves the screen).

[0041] This embodiment then calculates the upper and lower envelopes of the signal based on the obtained coarse friction events, and the difference between them changes over time. The difference increases as friction noise is present, and approaches zero in segments without sound. Therefore, based on this change, the start and end points of each friction noise segment can be accurately detected.

[0042] The specific algorithm process of the triboacoustic cutting algorithm based on signal envelope in this embodiment is described below:

[0043] First, a threshold T is set for the difference between the upper and lower envelopes of the signal. dif And a threshold T is set for the duration of each friction sound segment. dur When the difference between the upper and lower envelopes becomes greater than T dif At that time, the signal point may be the starting point of the frictional sound. Similarly, when the difference becomes less than T... dif At this point, the signal point may be the end of the friction sound. Finally, the duration of the friction sound needs further verification. Only when the time interval between the start and end signal points is greater than T... dur Only when the volume is high will this sound signal be considered a valid friction sound. After extensive testing, T was adjusted.dif and T dur The value can accurately cut out each friction sound. If the number of friction sounds is inconsistent with the number of line segments in the pattern, the input is considered invalid and the unlock pattern needs to be redrawn.

[0044] Step 5: Extract fingerprint features and input them into the identity verification model. For each friction sound, a confidence score is given. After weighted processing, users whose scores are greater than the preset value are considered legitimate users.

[0045] The friction sound generated by the relative sliding of a user's finger and the surface of a mobile device screen is mainly dependent on the user's fingerprint characteristics. Based on this finding, this embodiment extracts eight signal features from coarse to fine, resulting in a total of 99-dimensional feature vectors, including: a 7-dimensional spectral descriptor, a 1-dimensional harmonic ratio, a 39-dimensional MFCC correlation coefficient (including standard MFCC coefficients, first-order MFCC difference coefficients, and second-order MFCC difference coefficients), an 11-dimensional LPCC coefficient, a 27-dimensional RASTA-PLP coefficient, a 12-dimensional LSF coefficient, a 1-dimensional kurtosis, and a 1-dimensional skewness.

[0046] In this embodiment, the extracted fingerprint features are input into the identity verification model for security authentication; when a user sets a gesture password, only the friction sound of a legitimate user can be collected. Based on this data, different identity verification models are trained for different legitimate users.

[0047] The authentication model in this embodiment uses OC-SVM to construct the authentication model. The input of the model is the 99-dimensional feature vector of each friction sound segment. In this embodiment, the Gaussian radial basis function (RBF) is selected to train the authentication model and the Sequence Minimum Optimization (SMO) algorithm is selected as the optimization function.

[0048] After obtaining the friction sounds generated when a user draws an unlock pattern, the corresponding features are extracted and input into a pre-trained authentication model. The authentication model assigns a confidence score to each friction sound, and then applies an adaptive weighting strategy to obtain a weighted score. The higher the score, the higher the probability that the user is a legitimate user.

[0049] The specific implementation process of the adaptive weighting strategy in this embodiment is explained as follows:

[0050] When a user's fingertip is perpendicular to the swipe direction when drawing a pattern, the resulting friction sound better reflects the user's fingerprint characteristics compared to when it's parallel. Based on this principle, different authentication weights are assigned to the friction sounds corresponding to straight lines with different swipe directions. Lines that users prefer to draw with their fingertip perpendicular to the swipe direction are given higher weights. Then, the authentication confidence scores of the friction sounds corresponding to each straight line segment in a pattern are weighted and summed to obtain a weighted total score S. Only when S >= T... leg Only when T is reached will the unknown user be considered a legitimate user. leg These are empirical thresholds obtained through multiple tests. During use, the system adaptively adjusts the weights based on the results of multiple authentications. If a line segment with a higher weight consistently has a lower friction sound score, the system will decrease its weight. Conversely, if a line segment with a lower weight consistently has a higher friction sound score, the system will increase its weight.

[0051] When attempting brute-force attacks by traversing patterns, or making multiple attempts to crack a known correct pattern password, if the number of times the system continuously identifies the user as an unauthorized user exceeds a set value (e.g., 5 times), the system will lock the device and switch the input method to a complex password.

[0052] This invention utilizes the different friction sounds produced by the user's fingers and the screen when entering a gesture password to study and explore a secure gesture password authentication mechanism based on friction sound that can implicitly authenticate the user's fingerprint and prevent shoulder spying attacks.

[0053] When a user authenticates, they enter a gesture password, while the built-in microphone simultaneously captures the sound of the gesture. Specifically, the gesture password pattern must pass through at least four different points, a common setting in most commercial smartphones today. After the user completes the gesture password input, the entered pattern is matched against patterns stored in the system during registration. If the gesture password is incorrect, the user is immediately rejected. Otherwise, the client transmits the real-time recorded audio to the server for further processing.

[0054] Current methods by which attackers obtain users' gesture passwords include shoulder spying attacks (observing users directly, using mirrors, or hidden pinhole cameras to spy on their password input), sensor attacks (analyzing smartphone accelerometer data to infer unlock patterns), oil smudge attacks (reconstructing unlock patterns using smudges created when users draw patterns on the screen), eye attacks (inferring unlock patterns by observing involuntary eye movements during unlocking), and acoustic reflection attacks (inferring unlock patterns by observing sound signals reflected from fingertips). This invention provides effective defense against all of these attack methods.

[0055] It should be noted that in practical applications, the friction sound produced by a user's fingers rubbing the screen is relatively small, making it difficult for attackers to record the sound through a concealed microphone for replay attacks. Since each gesture password pattern has multiple segments, and corresponding friction sounds also have multiple segments, it is necessary to consider how to effectively defend against attacks while ensuring a good user experience.

[0056] It should be understood that the above description of the preferred embodiments is quite detailed, but it should not be considered as a limitation on the scope of protection of this invention. Those skilled in the art, under the guidance of this invention, can make substitutions or modifications without departing from the scope of protection of the claims of this invention, and all such substitutions or modifications fall within the scope of protection of this invention. The scope of protection of this invention should be determined by the appended claims.

Claims

1. A secure gesture password authentication method based on friction sound, characterized in that, Includes the following steps: Step 1: The user draws an unlock pattern on the screen with their finger. The mobile device's built-in microphone captures the friction sound and obtains the friction sound information. Step 2: Match the unlock pattern drawn by the user with the unlock patterns stored in the system during registration; If the pattern is incorrect, the system will immediately report that the user is an unauthorized user; otherwise, the recorded audio will be processed further. Step 3: Friction sound preprocessing, including background noise removal, target signal enhancement and wavelet re-denoising, to obtain a clean and enhanced sound signal; Step 4: Detect the friction time and cut off the friction sound; The probability of the presence of frictional sound at different times is obtained using a speech activity detection algorithm based on a hidden Markov model. Friction sounds with a probability greater than a preset value are considered as coarse friction events; the friction sounds are accurately segmented by calculating the upper and lower envelopes of the friction event signals. Step 5: Extract fingerprint features and assign a confidence score to each friction sound. After weighted processing, users whose scores are greater than the preset value are considered legitimate users.

2. The secure gesture password authentication method based on friction sound according to claim 1, characterized in that: In step 3, the background noise cancellation process involves selecting a Butterworth bandpass filter to acquire the 5-22 kHz friction sound signal and using a third-order Butterworth filter to maintain the smooth amplitude-frequency characteristics of the passband.

3. The secure gesture password authentication method based on friction sound according to claim 1, characterized in that: The target signal enhancement process described in step 3 uses multi-band spectral subtraction to filter out colored noise at different frequencies that affect the spectral uniformity in the audio after background noise elimination, thereby enhancing the speech spectrum. Then, the friction sound spectrum is divided into four non-overlapping frequency bands, and the posterior signal-to-noise ratio and corresponding subtraction coefficients of each frequency band are estimated.

4. The secure gesture password authentication method based on friction sound according to claim 1, characterized in that: The wavelet re-denoising process described in step 3 first decomposes the signal using the maximum overlap discrete wavelet transform method; then, it further processes the signal using a threshold denoising method, where the threshold is adjusted based on noise estimation at different levels, and soft thresholding is applied to the detail coefficients; finally, the wavelet coefficients obtained after processing are reconstructed using the inverse maximum overlap discrete wavelet transform method to obtain the denoised audio signal.

5. The secure gesture password authentication method based on friction sound according to claim 1, characterized in that: In step 5, the extracted fingerprint features are input into the identity verification model for security authentication. The identity verification model is constructed using OC-SVM, and the input of the model is the 99-dimensional feature vector of each friction sound. The identity verification model is trained using a Gaussian kernel, and the sequence minimum optimization algorithm is used as the optimization function.

6. The secure gesture password authentication method based on friction sound according to claim 1, characterized in that: In step 5, eight acoustic signal features were extracted from coarse to fine, resulting in a total of 99-dimensional feature vectors, including: a 7-dimensional spectral descriptor, a 1-dimensional harmonic ratio, a 39-dimensional MFCC correlation coefficient, an 11-dimensional LPCC coefficient, a 27-dimensional RASTA-PLP coefficient, a 12-dimensional LSF coefficient, a 1-dimensional kurtosis, and a 1-dimensional skewness; the MFCC correlation coefficient includes standard MFCC coefficients, first-order MFCC difference coefficients, and second-order MFCC difference coefficients.

7. The secure gesture password authentication method based on friction sound according to any one of claims 1-6, characterized in that: In step 5, when attempting brute-force attacks by traversing patterns or making multiple attempts to crack a known correct pattern password, if the number of consecutive authentication failures exceeds 5, the system will lock the device and switch the input method to a complex password.

8. A secure gesture password authentication system based on friction sound, characterized in that, Includes the following modules: Module 1 is used by users to draw an unlock pattern on the screen with their fingers. The mobile device's built-in microphone captures the friction sound and obtains the friction sound information. Module 2 is used to match the unlock pattern drawn by the user with the unlock patterns stored in the system during registration; If the pattern is incorrect, the system will immediately report that the user is an unauthorized user; otherwise, the recorded audio will be processed further. Module 3 is used for friction sound preprocessing, employing background noise reduction processing, target signal enhancement processing, and wavelet re-denoising processing to obtain a clean and enhanced sound signal; Module 4 is used to detect friction time and cut the friction sound; The probability of the presence of frictional sound at different times is obtained using a speech activity detection algorithm based on a hidden Markov model. Friction sounds with a probability greater than a preset value are considered as coarse friction events; the friction sounds are accurately segmented by calculating the upper and lower envelopes of the friction event signals. Module 5 is used to extract fingerprint features and give a confidence score for each friction sound. After weighted processing, users whose scores are greater than the preset value are considered legitimate users.

9. A secure gesture password authentication device based on friction sound, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the friction sound-based secure gesture password authentication method as described in any one of claims 1 to 7.

Images