A continuous identity authentication method based on sliding vibration signals

Abstract

The present invention relates to a continuous identity authentication method based on sliding vibration signals, and belongs to the field of mobile computing application technology. When the touch screen detects the sliding action of the human body, the vibration motor generates a specific vibration signal, and the accelerometer receives a unique vibration response affected by the sliding of the finger. Since each person's fingers have unique characteristics such as shape, size, bone density and muscle distribution, a vibration response with unique individual differences will be generated. This method designs a new vibration signal generation mechanism, including a combination of two different types of vibration signals and the addition of random frequency components. The present invention extracts different user biometrics from the two received signals, and designs a neural network to eliminate the influence of behavioral characteristics in the user's biometrics. Finally, continuous user identity authentication is achieved by utilizing biometrics that are unrelated to behavior. The present invention has low cost, strong anti-interference ability, high security, and is suitable for most relevant scenarios.

Description

Technical Field

[0001] The present invention relates to a biometric identity authentication method, specifically a method that uses a vibration motor and accelerometer built into a smart device to sense the biometric characteristics of a user's finger sliding and uses the biometric characteristics to perform continuous identity authentication, belonging to the field of mobile computing application technology. Background Art

[0002] With the advent of the mobile internet era, a wide variety of mobile smart devices (such as smartphones, smart bracelets, and smart watches) have become widely used in people's daily lives. As the computing and storage capabilities of mobile smart devices continue to increase, the amount of personal privacy data stored in them is also increasing. This includes many sensitive areas involving user privacy information, such as social communication, quick payment, and health monitoring. Relevant research shows that nearly 90% of users are concerned about the security of their private data. To prevent criminals from gaining unauthorized access to mobile smart devices and misappropriating users' private data, a reliable user identity authentication system is essential.

[0003] Currently, common mobile smart devices include smartphones and smart wristbands. These devices typically use knowledge-based authentication methods, where users set and memorize passwords or pattern locks for authentication. This method is inexpensive to implement and easy to deploy on a large scale across smart devices. However, traditional knowledge-based user authentication methods lack security against shoulder surfing and smear attacks, posing a risk of data leakage.

[0004] To overcome these shortcomings, methods that leverage the body's unique biometrics for identity authentication are widely used on smart devices, such as face recognition, fingerprint recognition, and voice recognition. Unlike traditional knowledge-based authentication methods, biometric-based authentication is theoretically more secure due to the uniqueness of user biometrics.

[0005] However, authentication methods based on user biometrics still have some problems. First, biometrics can be easily stolen by attackers and used to launch replay attacks on the authentication system. Second, because users only perform access authentication once, when unlocking their smart devices, identity verification is not performed again during subsequent use. If the user forgets to lock the system, attackers can steal private data.

[0006] To continuously authenticate users currently operating smart devices, some researchers have proposed methods for continuous authentication. For example, some methods implement continuous user authentication based on the characteristics of different interactions between users and the smart device's touchscreen. However, authentication systems that are highly dependent on the consistency of user behavior face significant threats, as attackers can infiltrate the system by learning and mimicking the behavior of legitimate users. Other methods combine smartphones' inertial measurement units and cameras to capture and calibrate facial images to implement continuous authentication systems. However, these systems are sensitive to ambient lighting conditions during use.

[0007] In summary, there is an urgent need for a more secure, convenient and reliable continuous user authentication method. Summary of the Invention

[0008] The purpose of the present invention is to address the shortcomings and defects of the existing technology and to solve the technical problem of continuously authenticating the operation of smart devices with high security and user-friendliness. A continuous identity authentication method that can be deployed on smart devices and senses the biometric characteristics of the user's sliding fingers based on active vibration signals is creatively proposed.

[0009] The innovation of this invention lies in that when the touchscreen of a smart device detects a human finger sliding, the vibration motor generates a specific vibration signal, and the accelerometer receives the unique vibration response affected by the finger sliding. Because each person's fingers have unique characteristics such as shape, size, bone density, and muscle distribution, they will produce vibration responses with unique individual differences. This method designs a new vibration signal generation mechanism, which involves combining two different types of vibration signals and adding a random frequency component (RFS).

[0010] The present invention extracts different user biometric features from the two received signals and designs a neural network to eliminate the influence of behavioral features on the user's biometric features. Finally, continuous user identity authentication is achieved by utilizing biometric features that are unrelated to behavior.

[0011] The objectives of the present invention are achieved through the following technical solutions.

[0012] A continuous identity authentication method based on sliding vibration signals comprises the following steps:

[0013] Step 1: The smart device generates and collects active vibration signals.

[0014] In order to extract the biometric features of human fingers for authentication, a vibration motor is built into the mobile smart device to generate active vibration signals.

[0015] Since human fingers have different effects on different types of vibration signals, the active vibration signal designed in the present invention combines two different types of vibration signals to obtain the user's finger biometric characteristics, including a linear frequency modulation signal and a stable signal.

[0016] Specifically, step 1 includes the following steps:

[0017] Step 1.1: Generate a vibration signal.

[0018] When a mobile smart device senses a finger sliding, the built-in vibration motor first generates a linear frequency modulation signal, whose vibration frequency increases linearly over time. After the linear frequency modulation signal is generated, the vibration motor continues to generate a stable signal (a cosine wave containing only a single frequency) until the user stops sliding their finger.

[0019] Step 1.2: Receive vibration signal.

[0020] When the vibration motor generates a vibration signal, an accelerometer on the same mobile smart device is used to receive the vibration response. Preferably, the sampling rate of the accelerometer can be set to 1000 Hz.

[0021] Step 1.3: Add random frequency components, referred to as RFS.

[0022] To further improve attack resistance, the present invention has designed a RFS mechanism. This mechanism randomly generates several short cosine wave signals with random frequencies during each user authentication. By comparing the RFS values ​​in the received and transmitted signals, the authenticity of the vibration response signal received by the accelerometer during this authentication is determined. This prevents attackers from replaying previously authenticated vibration response signals.

[0023] Step 2: Preprocess the received vibration response signal.

[0024] Specifically, step 2 includes the following steps:

[0025] Step 2.1: Select the coordinate axis containing the most obvious vibration response.

[0026] Since the vibration response signal collected by the accelerometer contains data on the X-axis, Y-axis, and Z-axis, it is necessary to select the data on the acceleration axis that is most sensitive to the vibration signal for subsequent processing.

[0027] The present invention uses the signal-to-noise ratio to define the sensitivity of each coordinate axis of the accelerometer to the vibration signal. The larger the signal-to-noise ratio of the vibration response signal to the noise signal, the greater the proportion of the vibration response power.

[0028] Step 2.2: Remove motion noise.

[0029] The vibration response signal received by the accelerometer includes not only the influence of the finger sliding motion but also the influence of the user's body movement. Therefore, it is necessary to filter out the motion noise in the vibration response of the selected accelerometer coordinate axis.

[0030] For example, a high-pass filter with a cutoff frequency of 100 Hz can be used to remove the interference of low-frequency motion noise.

[0031] Step 2.3: Segment the two vibration signals.

[0032] The present invention uses two vibration signals to capture different biometric features of a user's sliding finger. In order to process the two signals separately, it is necessary to first determine the segmentation point between the linear frequency modulation signal and the stable signal in the received signal.

[0033] First, the accelerometer signal is divided into several frequency bands, and the variance of the amplitude between the bands is calculated. Because the band variance stores information about the fluctuations between multiple frequency bands, there is a minimum band variance between the linear frequency modulation signal and the stable signal. The time corresponding to this minimum value is the dividing point between the two signal segments.

[0034] Step 3: Match the RFS in the transmitted and received signals.

[0035] During login authentication, RFS is added to the stable signal segment of the transmitted signal, and then it is determined whether the RFS in the received signal matches the RFS in the transmitted signal. The following steps are included:

[0036] Step 3.1: Time domain matching.

[0037] Time domain matching is to determine whether the start and end times of each RFS in the received signal and the transmitted signal are consistent.

[0038] Specifically, the short-term energy of the received signal is calculated, and an algorithm based on the short-term energy difference is used to detect the start and end times of each RFS. The time difference between the start / end time of the RFS and the start / end time of the RFS in the transmitted signal known to the system is then calculated. If the start / end time difference is less than a set value (for example, 5ms), the RFS of the transmitted and received signals are considered to match in the time domain.

[0039] Step 3.2: Frequency domain matching.

[0040] Frequency domain matching is to check whether the frequencies of RFS in the transmitted signal and the received signal are the same.

[0041] Specifically, a fast Fourier transform (FFT) is used to analyze the frequency distribution of stable signal segments. Each RFS frequency has relatively high energy, resulting in a high peak in the spectrum. If the difference in RFS frequency between the transmitted and received signals is less than a set value (e.g., 5 Hz), the RFS of the transmitted and received signals are considered matched in the frequency domain.

[0042] Step 4: Extract human biometric features from the vibration response signal received by the accelerometer.

[0043] Specifically, step 4 includes the following steps:

[0044] Step 4.1: Extract features from the linear FM signal.

[0045] In order to capture the human biological characteristics contained in the linear frequency modulation signal, the present invention adopts the synchronous compression wavelet transform algorithm. First, the linear frequency modulation signal is transferred from the time-scale plane to the time-frequency plane by using the wavelet transform, and then the energy of the plane is redistributed to obtain the time-frequency characteristics with a more concentrated frequency curve.

[0046] Step 4.2: Extract features from the stable signal.

[0047] The human biometrics contained in the stable signal are primarily distributed in the frequency band around 150Hz and its harmonic 300Hz. This paper uses the empirical wavelet transform to extract the biometric features from it. The Fourier spectrum of the stable signal is divided into continuous intervals. An orthogonal wavelet filter bank is then constructed in each interval to reconstruct the signal. This extracts a set of amplitude-frequency modulated (AM / FM) signals, which are then subjected to a Hilbert transform to obtain the instantaneous frequency and amplitude that can characterize the biometric features of human fingers.

[0048] Step 5: User authentication.

[0049] Step 5.1: Training set construction.

[0050] Specifically, the present invention uses a Triplet network to reconstruct the user's biometric characteristics, so that the processed user biometric characteristics only contain the physical biometric characteristics inherent in human fingers, eliminating the influence of behavioral characteristics (including sliding duration, sliding force and start and end positions).

[0051] The Triplet network consists of three identical sub-networks, each inputting a triplet consisting of a baseline sample, a positive sample, and a negative sample. Positive samples and baseline samples are samples of the same user with different behaviors, while negative samples and baseline samples are samples of the same behavior from different users. The output of each sub-network is a behavior-independent physical biometric feature vector.

[0052] Before training the network, we first construct an input triplet. To determine whether two swipes contain the same behavior, we obtain the behavioral features of each swipe from the touchscreen sensor. If the difference between the behavioral features of two swipes is greater than a set value, the two swipes are considered to have different behaviors.

[0053] Step 5.2: Network model training.

[0054] Specifically, each sub-network consists of four convolutional layers, three maximum pooling layers, one spatial pyramid pooling layer, and two fully connected layers. During training, the loss of the Triplet network is minimized to extract physical biometric features that are independent of user behavior.

[0055] Step 5.3: User registration and login.

[0056] When a new user registers, their extracted features and those of other registered users are used to form new triplet samples. The triplet input samples are used to fine-tune the parameters of the Triplet network. A unique central biometric signature is then generated for the new user, defined as the average of the behavior-independent physical biometric feature vectors extracted from the user's registration data.

[0057] When a user logs in, the Euclidean distance between the logged-in user's behavior-independent biometric and all central biometrics in the system is calculated. The currently logged-in user is then identified as the user whose central biometric corresponds to the minimum distance. A threshold is also set to distinguish legitimate users from attackers. If the minimum distance is greater than the threshold, the currently logged-in user is deemed illegitimate and access is denied.

[0058] Beneficial effects

[0059] Compared with the prior art, the present invention has the following advantages:

[0060] 1. The present invention relies solely on the vibration motor and accelerometer in the smart device to extract the biometric features of the user's sliding finger to continuously authenticate the user's identity.

[0061] 2. The present invention has low cost, strong anti-interference ability and high security. Since sliding operation is a common interaction method used in most smart phones and smart wristband devices, it is applicable to most scenarios.

[0062] 3. The present invention designs a transmission signal containing two different types of vibration signals, and uses different algorithms to extract different biometric features of the user from the two vibration signal components.

[0063] 4. The present invention designs a new RFS mechanism to further improve the system's anti-attack capability and effectively prevent attackers from conducting replay attacks.

[0064] 5. The present invention combines the Triplet network to extract the user's unique physical biometric features to achieve accurate authentication, while also avoiding the possibility of attackers imitating the actions of legitimate users to launch imitation attacks. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 This is a schematic diagram of the user authentication method according to an embodiment of the present invention.

[0066] Figure 2 This is a diagram of the Triplet network structure of an embodiment of the present invention.

[0067] Figure 3 This is the overall performance of the embodiment of the present invention.

[0068] Figure 4 This is the performance of the embodiment of the present invention under different registration sliding times.

[0069] Figure 5 The performance of the embodiment of the present invention under different vibration intensities is shown.

[0070] Figure 6 The performance of the embodiment of the present invention in different scenarios.

[0071] Figure 7 This is the performance of the embodiment of the present invention under replay attack.

[0072] Figure 8 This is the performance of the embodiment of the present invention under imitation attacks. DETAILED DESCRIPTION

[0073] The present invention is further described in detail below with reference to the embodiments and drawings.

[0074] A continuous identity authentication method based on sliding vibration signals, such as Figure 1 As shown, the following steps are included:

[0075] Step 1: The smart device generates and collects active vibration signals.

[0076] To extract the biometric characteristics of a user's finger for authentication, it is first necessary to design an active vibration signal that can be generated by the vibration motor of a mobile smart device. Most mobile smart devices today use linear vibration motors, which generate vibration signals with adjustable frequency and amplitude. Because fingers have different effects on different types of vibration signals, the active vibration signal designed in this invention combines two different types of vibration signals (i.e., a linear frequency modulation signal and a steady-state signal) to obtain the biometric characteristics of the user's finger.

[0077] Step 1.1: Generate a vibration signal.

[0078] Once the mobile smart device senses the sliding action of the user's finger, the linear vibration motor will first generate a linear frequency modulation signal, whose vibration frequency increases linearly with time. Generally speaking, the resonant frequency of the linear vibration motor is about 175Hz to 235Hz. Therefore, the present invention sets the frequency range of the linear frequency modulation signal to 150Hz to 250Hz. It was found through observation that the duration of most sliding is greater than 180ms. In order to ensure that the linear frequency modulation signal can be sent out completely, its duration is set to 150ms. After sending the linear frequency modulation signal, the vibration motor will continue to generate a stable signal (referring to a cosine wave containing only a single frequency) to capture the biometric features of the finger. The vibration frequency of the stable signal is 150Hz. The linear vibration motor will continue to generate a stable vibration signal until the user's finger stops sliding.

[0079] Step 1.2: Receive vibration signal.

[0080] When the linear vibration motor begins generating a vibration signal, an accelerometer on the same mobile smart device is used to receive the vibration response. The accelerometer's sampling rate is set to 1000 Hz, which is within the upper limit of the sampling rate supported by existing mobile smart devices.

[0081] Step 1.3: Add RFS.

[0082] In order to further improve the anti-attack performance, the present invention designs an RFS mechanism, that is, each time a user authenticates, several cosine wave signals with random frequencies and very short durations are randomly generated and superimposed on the stable signal segment.

[0083] Specifically, in this embodiment, the duration of each RFS is 30ms, the frequency range of each RFS is set to [100,135]∪[165,250]Hz, the number of RFSs in the vibration signal sent for each authentication does not exceed 3, and the time interval between any two RFSs is greater than 30ms, thereby ensuring the distinguishability between RFSs.

[0084] By comparing the RFS in the received signal with the RFS in the transmitted signal, the authenticity of the vibration response signal received by the accelerometer during this authentication is determined. Therefore, an attacker cannot use a previously stolen vibration response signal to perform a replay attack.

[0085] Step 2: Preprocess the received vibration response signal.

[0086] After the accelerometer receives the transmitted signal, it first needs to perform preliminary preprocessing on the received vibration response signal.

[0087] Step 2.1: Select the coordinate axis containing the most obvious vibration response.

[0088] Since the vibration response signal collected by the accelerometer contains data from the X-axis, Y-axis, and Z-axis, it is necessary to select the data on the acceleration axis that is most sensitive to the vibration signal for subsequent processing. The present invention uses the signal-to-noise ratio to define the sensitivity of each coordinate axis of the accelerometer to the vibration signal. The greater the signal-to-noise ratio of the vibration response signal to the noise signal, the greater the proportion of the vibration response power. Since the accelerometer usually collects data continuously for other system applications such as pedometers, the acceleration data within 1 second before the user slides is regarded as a noise signal. Afterwards, the vibration response of the most sensitive coordinate axis is selected for processing.

[0089] Step 2.2: Motion noise removal.

[0090] The vibration response signal received by the accelerometer includes not only the effects of finger sliding but also the user's body movements. Therefore, it is necessary to filter out motion noise from the vibration response of the selected accelerometer coordinate axis. Analysis has found that the vibration frequency generated by daily activities is mostly below 80Hz, while the biometric characteristics of a user's finger sliding are primarily extracted from vibration response signals with frequencies above 150Hz. Therefore, a high-pass filter with a cutoff frequency of 100Hz can be used to remove the interference of low-frequency motion noise.

[0091] Step 2.3: Segment the two vibration signals.

[0092] The present invention uses two vibration signals to capture the different biometric characteristics of a user's sliding finger. To process these two signal segments separately, the split point between the linear frequency modulation signal and the stable signal in the received signal must be determined. The energy of the linear frequency modulation signal varies greatly with its frequency, while the energy of the stable signal is generally concentrated in a very small frequency band. Therefore, the signal received by the accelerometer is first divided into several frequency bands, and the variance of the amplitude between the frequency bands is calculated. Because there is a minimum value of the frequency band variance between the linear frequency modulation signal and the stable signal, the time corresponding to this value is the split point between the two signal segments. Therefore, the present invention determines the location of the split point by searching for the minimum value of the variance.

[0093] Step 3: Match the RFS in the transmitted and received signals.

[0094] During authentication, an RFS with a random start time and frequency is added to the stable signal segment. The RFS in the received signal then needs to be checked to see if it matches the RFS in the transmitted signal. If they have the same time-frequency information, the received signal is indeed generated during the authentication process, not a replayed attack signal.

[0095] Step 3.1: Time domain matching.

[0096] Time domain matching determines whether the start and end times of each RFS in the received and transmitted signals are consistent. Because the energy of a signal segment with an RFS is significantly higher than that without one, short-term energy is used to determine the start and end times of each RFS.

[0097] Specifically, in this embodiment, a sliding window with a length of 10ms is applied, sliding 2ms on the signal each time, and calculating the short-time energy E of the signal in the t-th window t .

[0098] Then, an algorithm based on short-time energy difference is used to detect the start and end time of each RFS. t =E t+1 -E t Calculate the short-time energy difference D between the two windows t .

[0099] Set the thresholds of start time and end time to ET respectively s and ET e In the tth window, D t >ET s And E t+2 >E t+1 , take t as the starting time of the candidate RFS. In the t′th window, D t′ <ET e And E t′ <E t′-1 , taking t′ as the end time of the candidate RFS. Then, continue to search for the start and end time of the next candidate RFS until the start and end times of all candidate RFSs are found. After that, calculate the energy peak of each candidate RFS and select the top k RFSs with the largest peaks (k is the number of RFSs in the transmitted signal). Then, calculate the time difference between the start (end) time of the selected RFS and the start (end) time of the RFS in the transmitted signal known to the system. If the start (end) time difference is less than 5ms, the RFS of the transmitted and received signals are considered to match in the time domain.

[0100] Step 3.2: Frequency domain matching.

[0101] Frequency domain matching, that is, checking whether the frequencies of the RFS in the transmitted signal and the received signal are the same. The present invention uses fast Fourier transform to analyze the frequency distribution of the stable signal segment. The highest peak on the spectrum is the transmitting frequency of the stable signal, 150Hz. The frequency energy of the RFS is relatively high, so it will appear as a higher peak in the spectrum. The peak detection algorithm is used to obtain all the peaks in the spectrum and sort them, and the k peaks with the largest remaining amplitudes except the highest peak are found (k is the number of RFS in the transmitted signal), and the corresponding frequencies of the k RFS can be obtained. Finally, the frequency difference between the frequencies of the selected k RFS and the frequency of the RFS in the transmitted signal is calculated. If the frequency difference is less than 5Hz, it is considered that the RFS of the transmitted signal and the received signal are matched in the frequency domain.

[0102] Step 4: Extract user biometrics from the vibration response signal.

[0103] Step 4.1: Extract features from the linear FM signal.

[0104] To capture the user's biometric characteristics contained in linear frequency modulation signals, the present invention employs a synchronous compression wavelet transform algorithm. This algorithm redistributes the energy of the time-scale plane based on the magnitude of the modulus of each element in the plane. It then transforms the time-scale plane into the time-frequency plane through a special mapping process, resulting in a more concentrated time-frequency feature curve.

[0105] For a linear frequency modulation signal, first calculate its continuous wavelet transform result as W(a,b), where a is the scale factor and b is the shift factor. Then, use phase transformation to extract the instantaneous frequency Ω(a,b). After obtaining the instantaneous frequency, transfer the information from the time-scale plane to the time-frequency plane. Let Ω l Represents the frequency closest to the origin Ω(a,b), and redistributes each value in the result of the continuous wavelet transform to T(Ω l ,b), the result of synchronous compression wavelet transform T(Ω l ,b):

[0106]

[0107] Where ΔΩ=Ω l -Ω l-1 , (Δa) k =a k -a k-1 , a k is the discrete wavelet scale.

[0108] Step 4.2: Extract features from the stable signal.

[0109] The user's biometric features contained in the stable signal are primarily distributed in the frequency band around 150 Hz and its harmonic 300 Hz. This invention uses the empirical wavelet transform to extract biometric features from this signal. The basic idea of ​​the empirical wavelet transform is to divide the Fourier spectrum of the signal into continuous intervals. Then, a suitable orthogonal wavelet filter bank is constructed in each interval to reconstruct the signal, extracting a set of amplitude-frequency modulated signals. Hilbert transform is then performed to obtain meaningful instantaneous frequency and amplitude.

[0110] Step 4.3: Feature integration.

[0111] After extracting the user biometric features from the linear frequency modulation signal segment and the stable signal segment, the biometric feature matrices of the two signal segments are concatenated together to form a complete biometric feature matrix.

[0112] Step 5: User authentication.

[0113] Step 5.1: Construct a training set.

[0114] The present invention uses a Triplet network to reconstruct the user's biometric characteristics, so that the processed user biometric characteristics only contain the physical biometric characteristics inherent in the user's fingers, while removing the influence of behavioral characteristics (including sliding duration, sliding force and start and end positions).

[0115] Specifically, the Triplet network consists of three identical sub-networks, whose input is a triplet consisting of a reference sample, a positive sample, and a negative sample.

[0116] For example, if the feature matrix of user U1 under behavior B1 is considered as the baseline sample, the feature matrix of user U1 under behavior B2 is a positive sample, and the feature matrix of user U2 under behavior B1 is a negative sample. The output of each sub-network is a physical biological feature vector that is unrelated to behavior.

[0117] The training goal of the Triplet network is to minimize the Euclidean distance between the output feature vectors of the reference sample and the positive sample, and to maximize the Euclidean distance between the output feature vectors of the reference sample and the negative sample. Before training the network, the input triples must be constructed.

[0118] To determine whether two swipes can be considered the same, we first obtain the duration, average force, and start and end positions of each swipe from the touchscreen sensor. We then calculate the difference between the two swipe durations. If the difference is greater than a threshold, we consider the two swipes to be different. Similarly, we perform the same operation for the average force and start and end positions. This method constructs a sample set of input triplets for training the Triplet network.

[0119] Step 5.2: Train the network model.

[0120] Each sub-network mainly consists of 4 convolutional layers, 3 maximum pooling layers, 1 spatial pyramid pooling layer and 2 fully connected layers, such as Figure 2 As shown in Figure 2. To prevent overfitting, a batch normalization layer is added after each max pooling layer. By minimizing the triplet network loss during training, physical biometric features that are independent of user behavior can be extracted.

[0121] Before system deployment, the Triplet network must be pre-trained. Three volunteers were asked to slide their fingers across different areas of a mobile device's screen with varying force, with the device positioned in two different positions (on a table or held in hand). After collecting data and extracting their biometric matrix, triplet samples were constructed as described above and fed into the Triplet network for pre-training. The pre-trained Triplet network then demonstrated the ability to extract behaviorally independent biometrics from different users.

[0122] Step 5.3: User registration and login.

[0123] When a new user registers, several swipes are collected as registration data. The features of the newly registered user and the features of other registered users (if any) are used to form new triplet samples. Based on the idea of ​​incremental learning, only the new triplet input samples are used to fine-tune the parameters of the Triplet network without completely retraining all parameters. A unique central biometric feature (CBF) is then generated for the newly registered user. It is defined as the average of the behavior-independent physical biometric feature vectors extracted from the user's registration data.

[0124] Login authentication includes the initial login authentication when unlocking a mobile smart device and continuous authentication during the user's interaction with the mobile device. Assuming there are x registered users, there are x corresponding CBFs in the system database. When a user logs in, the system first calculates x Euclidean distances between the behavior-independent feature vector extracted from the currently logged-in user and all CBFs in the system. Then, the currently logged-in user is identified as the user to whom the CBF corresponding to the minimum distance among the x distances belongs. At the same time, a threshold is set to distinguish between legitimate users and illegal users. If the minimum distance among the x distances is greater than the threshold, the system considers the currently logged-in user to be an illegal user and denies access.

[0125] Example

[0126] The present invention has implemented prototypes on different mobile smart devices (including smartphones and smart watches). The experiments were conducted in three scenarios with different motion noise levels, including when the user was sitting in the office, walking outdoors, and riding in a car. A total of 28 volunteers (aged from 21 to 49) were recruited to participate in the experiment, of which 20 (12 men and 8 women) served as legitimate users and the other 8 served as attackers. The 20 legitimate users were required to slide their fingers in 7 different sliding positions and using 3 different sliding forces in 3 scenarios. The 8 attackers performed 2 types of attacks on each legitimate user under the same conditions.

[0127] First, the overall performance of the present invention was evaluated. Figure 3 The present invention shows that 20 legitimate users (denoted as U1, U2, ..., U 20 ) and eight attackers (denoted as AT). The results show that the present invention achieves an average authentication accuracy of 95.7% for legitimate users and 99.5% for attackers. The above analysis demonstrates that the present invention can authenticate legitimate users with high accuracy and detect illegitimate users attempting to deceive the authentication system.

[0128] Excessive swipes during the registration phase may lead to a poor user experience. Subsequently, the false rejection rate when users swipe to register with different numbers of times was evaluated. Figure 4 As shown in the figure, the false rejection rate initially decreases rapidly as the number of swipes required for user registration increases. When the user swipes 6 times on a smartphone and 7 times on a smartwatch, the false rejection rate drops to 1.4% and 1.9%, respectively. To achieve a balance between system performance and user experience, the number of swipes required for user registration on smartphones is fixed at 6, and the number of swipes required for registration on smartwatches is fixed at 7.

[0129] Since active vibration signals are used to achieve user identity authentication, an experiment was subsequently conducted to evaluate the effect of vibration intensity on accuracy. The maximum vibration intensity of the linear vibration motor was defined as 100%, and the F1-scores under different vibration intensities were as follows: Figure 5 As shown in the figure, the F1-scores of the smartphone and smartwatch reached their maximum values ​​when the vibration intensity reached 25% and 30%, respectively. To achieve better system performance, the experiment set the active vibration signal intensity on the smartphone and smartwatch to 25% and 30%, respectively. User research also found that these two vibration intensities did not affect users' normal use of mobile smart devices in daily situations.

[0130] The experiments then evaluated the system performance under three common usage scenarios for users, including sitting in an office, walking outdoors, and riding in a car. Figure 6 The F1-scores for identity authentication in these three scenarios are shown. The highest F1-score is 96.4% when the user is sitting in an office. The lowest F1-score is 92.6% when the user is sitting in a car. Overall, the F1-scores in various scenarios are consistently above 92%, demonstrating that the present invention can maintain good user authentication results in a variety of scenarios and is environmentally robust.

[0131] To conduct a replay attack, the attacker places his smartphone and the legitimate user's smartphone on the same table and uses an accelerometer to record the legitimate user's vibration signal during the authentication process at different distances. The signal is then replayed to the authentication system for attack. This experiment also evaluates the ability to resist replay attacks when the RFS mechanism is not configured. The results are as follows: Figure 7 As shown in the figure, without the RFS mechanism, the false acceptance rate is initially significantly higher. However, as the distance between smartphones increases, the false acceptance rate gradually decreases, reaching 1.1% at a distance of 40 cm. However, with the RFS mechanism in place, the false acceptance rate remains below 0.5% at all distances, demonstrating that the RFS mechanism designed in this invention effectively enhances resistance to replay attacks.

[0132] For imitation attacks, this experiment assumes that the attacker can secretly observe and imitate the finger sliding behavior of legitimate users when logging into the system in different scenarios. Figure 8 The false acceptance rate (FAR) of an attacker performing impersonation attacks on smartphones and smartwatches in three usage scenarios is shown. The average FAR across different mobile smart devices is less than 1.3%, demonstrating that the present invention effectively protects against impersonation attacks in a variety of scenarios. This is because by extracting physical biometrics that are unrelated to user behavior, the impact of varying user sliding behavior on authentication accuracy is significantly reduced. Even if an attacker closely mimics the legitimate user's behavior, they do not possess the exact same physical biometrics as the legitimate user.

[0133] The specific examples described above are further explanations of the present invention and are not intended to limit the scope of protection of the present invention. Any changes and equivalent substitutions made within the principles and spirit of the present invention should be within the scope of protection of the present invention.

Claims

1. A continuous identity authentication method based on sliding vibration signals, characterized in that: The following steps are involved: Step 1: The smart device generates and collects active vibration signals; A vibration motor is built into the mobile smart device to generate an active vibration signal. The active vibration signal is combined with two different types of vibration signals to obtain the user's finger biometric characteristics, including a linear frequency modulation signal and a stable signal. Step 1.1: Generate a vibration signal; When a mobile smart device senses a finger sliding, the vibration motor first generates a linear frequency modulation signal, whose vibration frequency increases linearly over time. After the linear frequency modulation signal is generated, the vibration motor continues to generate a stable signal until the user stops sliding their finger. Step 1.2: Receive vibration signal; When the vibration motor generates a vibration signal, an accelerometer on the same mobile smart device is used to receive the vibration response; Step 1.3: Add random frequency components, referred to as RFS; During each user authentication, several cosine wave signals with random frequencies and very short durations are randomly generated. The authenticity of the vibration response signal received by the accelerometer during this authentication is determined by comparing the RFS in the received signal with the RFS in the transmitted signal. This prevents attackers from replaying previously authenticated vibration response signals. Step 2: Preprocess the received vibration response signal; Step 2.1: Select the coordinate axis containing the most obvious vibration response; The vibration response signal collected by the accelerometer includes data from the X-axis, Y-axis, and Z-axis. The data on the acceleration axis most sensitive to the vibration signal is selected for subsequent processing. The signal-to-noise ratio is used to define the sensitivity of each coordinate axis of the accelerometer to the vibration signal. The larger the signal-to-noise ratio of the vibration response signal to the noise signal, the greater the proportion of vibration response power. Step 2.2: Remove motion noise; Filter out motion noise in the vibration response of the selected accelerometer axes; Step 2.3: Segment the two vibration signals; Determine the split point between the linear frequency modulation signal and the stable signal in the received signal. First, divide the signal received by the accelerometer into several frequency bands and calculate the variance of the amplitude between the frequency bands. The time corresponding to the minimum value of the frequency band variance between the linear frequency modulation signal and the stable signal is the split point between the two signals. Step 3: Match the RFS in the transmitted and received signals; During login authentication, RFS is added to the stable signal segment of the transmitted signal, and then it is determined whether the RFS in the received signal matches the RFS in the transmitted signal. Step 3.1: Time domain matching; Time domain matching is to determine whether the start and end times of each RFS in the received signal and the transmitted signal are consistent; Calculate the short-time energy of the received signal and use an algorithm based on the short-time energy difference to detect the start and end time of each RFS; Then, the time difference between the start / end time of the RFS and the start / end time of the RFS in the transmitted signal known to the system is calculated; if the start / end time difference is less than the set value, the RFS of the transmitted signal and the received signal are considered to match in the time domain; Step 3.2: Frequency domain matching; Frequency domain matching is to check whether the frequencies of RFS in the transmitted and received signals are the same; Use fast Fourier transform to analyze the frequency distribution of stable signal segments. Each RFS has relatively high frequency energy, resulting in a higher peak in the spectrum. If the RFS frequency difference between the transmitted and received signals is less than a set value, the RFS of the transmitted and received signals are considered matched in the frequency domain. Step 4: Extracting human biometric features from the vibration response signal received by the accelerometer; Step 4.1: Extract features from the linear FM signal; The synchronous compression wavelet transform algorithm is adopted. First, the linear frequency modulation signal is transferred from the time-scale plane to the time-frequency plane by wavelet transform, and then the energy of the plane is redistributed to obtain a more concentrated time-frequency feature of the frequency curve. Step 4.2: Extract features from the stable signal; Use empirical wavelet transform to extract biometric features from it, divide the Fourier spectrum of the stable signal into continuous intervals, then construct an orthogonal wavelet filter bank on each interval to reconstruct the signal and extract a set of amplitude-frequency modulated signals. Then perform Hilbert transform to obtain the instantaneous frequency and instantaneous amplitude that can characterize the biometric features of human fingers; Step 5: User identity authentication; Step 5.1: Construction of training set; A Triplet network is used to reconstruct the user's biometrics, so that the processed user biometrics only contain the inherent physical biometrics of human fingers; The Triplet network consists of three identical sub-networks, whose input is a triple consisting of a baseline sample, a positive sample, and a negative sample. The positive sample and the baseline sample are samples of the same user with different behaviors, while the negative sample and the baseline sample are samples of different users with the same behavior. The output of each sub-network is a physical biometric feature vector that is independent of behavior. Before training the network, an input triplet is first constructed. The behavioral features of each sliding action are obtained from the touch screen sensor. If the difference between the behavioral features of two sliding actions is greater than a set value, the two sliding actions are considered to be different. Step 5.2: Network model training; During the training process, the loss of the Triplet network is minimized to extract physical biometric features that are independent of user behavior. Step 5.3: User registration and login; When a new user registers, their extracted features and those of other registered users are used to form new triplet samples. The parameters of the Triplet network are fine-tuned using the triplet input samples. A unique central biometric signature is then generated for the new user, which is defined as the average of the behavior-independent physical biometric feature vectors extracted from the user's registration data. When a user logs in, the Euclidean distance between the logged-in user's behavior-independent biometric features and all central biometric features in the system is first calculated; then, the currently logged-in user is identified as the user belonging to the central biometric feature corresponding to the minimum distance; at the same time, a threshold is set to distinguish between legitimate users and attackers. If the minimum distance is greater than the threshold, the currently logged-in user is considered not a legitimate user and their access is denied.

2. A continuous identity authentication method based on sliding vibration signals as claimed in claim 1, characterized in that: In step 1.2, the sampling rate of the accelerometer is set to 1000 Hz.

3. The continuous identity authentication method based on sliding vibration signals according to claim 1, characterized in that: In step 2.2, a high-pass filter with a cutoff frequency of 100 Hz is used to remove the interference of low-frequency motion noise.

4. The continuous identity authentication method based on sliding vibration signals according to claim 1, characterized in that: In step 5.2, each sub-network of the network model includes 4 convolutional layers, 3 maximum pooling layers, 1 spatial pyramid pooling layer and 2 fully connected layers.

5. The continuous identity authentication method based on sliding vibration signals according to claim 1, characterized in that: In step 1.3, the duration of each RFS is 30 ms, the frequency range of each RFS is set to [100, 135] ∪ [165, 250] Hz, the number of RFSs in the vibration signal sent for each authentication does not exceed 3, and the time interval between any two RFSs is greater than 30 ms.

6. The continuous identity authentication method based on sliding vibration signals according to claim 1, characterized in that: In step 3.1, first, according to D t =E t+1 -E t Calculate the short-time energy difference D between the two windows t ; Set the start time and end time thresholds to ET s and ET e ; In the tth window, D t >ET s And E t+2 >E t+1 , take t as the starting time of the candidate RFS; in the t′th window, D t′ <ET e And E t′ <E t′-1 , take t′ as the end time of the candidate RFS; then, continue to look for the start and end time of the next candidate RFS until the start and end time of all candidate RFS are found; After that, the energy peak of each candidate RFS is calculated, and the top k RFSs with the largest peaks are selected, where k is the number of RFSs in the transmitted signal. Then, the time difference between the start / end time of the selected RFS and the start / end time of the RFS in the transmitted signal known to the system is calculated. If the start / end time difference is less than the set value, the RFS of the transmitted signal and the received signal are considered to match in the time domain.

7. The continuous identity authentication method based on sliding vibration signals according to claim 1, characterized in that: In step 4.1, for a linear frequency modulation signal, first calculate its continuous wavelet transform result as W(a,b), where a is the scale factor and b is the shift factor; then, use phase transformation to extract the instantaneous frequency Ω(a,b); after obtaining the instantaneous frequency, transfer the information from the time-scale plane to the time-frequency plane; let Ω l Represents the frequency closest to the origin Ω(a,b), and redistributes each value in the result of the continuous wavelet transform to T(Ω l ,b), the result of synchronous compression wavelet transform T(Ω l ,b): Where ΔΩ=Ω l -Ω l-1 , (Δa) k =a k -a k-1 , a k is the discrete wavelet scale.

Images