A two-factor authentication method based on acoustic signal of arm-raising gesture

Abstract

The application discloses a kind of based on acoustic signal's arm-lifting gesture two-factor identity authentication method, belong to information security and internet of things security field.The ultrasonic wave signal is emitted by ultrasonic wave emission device, and the signal reflected back in the arm-lifting action when card is brushed is captured using receiving device.The collected signal is processed, and arm-lifting gesture signal is effectively extracted.Improved LSTM-FCN model is used to extract signal characteristics, and high-performance and lightweight identity authentication model is constructed using SVM, and the training cost is reduced by transfer learning based on fine-tuning.Experiments prove that the detection accuracy of the method to arm-lifting signal is 97.191%, and the user recognition accuracy of the method is 98.540%.The method can further improve security, compared with existing two-factor authentication, with stronger universality and concealment, easier to promote.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of information security and Internet of Things security, and relates to a voice signal-based arm-lifting gesture two-factor identity authentication method. BACKGROUND

[0002] With the advancement of intelligence, smart cards have become an indispensable part of our lives, and authentication methods based on card swiping behaviors such as payment and unlocking can be seen everywhere. While simplifying identity verification and enhancing user experience, smart cards also face huge security threats. Loss of smart cards, leakage of card information, and unauthorized copying can all be exploited by attackers, causing huge losses to users. According to the annual report of the Nielsen Company, credit card fraud will cause a global loss of about 408 billion US dollars in the next 10 years. Therefore, there is an urgent need for a tool that can improve the security of card swiping behaviors.

[0003] Fortunately, various card swiping behaviors are often accompanied by arm-lifting actions, which are different for different individuals and difficult to imitate or copy. Therefore, selecting arm-lifting actions as the second factor of identity authentication can effectively improve the security of card swiping behaviors. Existing recognition of arm-lifting actions mainly includes recognition based on wearable devices and recognition based on cameras. Recognition based on wearable devices faces the disadvantages of additional cost of the second device, new threats caused by loss of authentication devices, and low concealment. Authentication based on cameras is often affected by environmental factors such as light and line of sight, so there is a great security risk. Therefore, there is an urgent need for a recognition method that is closer to real-life scenarios, easier to promote, and more stable to ensure user information security. SUMMARY

[0004] In view of the deficiencies of the prior art, a voice signal-based arm-lifting gesture two-factor identity authentication method is provided, which is more concealed, easier to promote, and safer.

[0005] To achieve the above technical purpose, the present application discloses a voice signal-based arm-lifting gesture two-factor identity authentication method, which simultaneously identifies a handheld smart card through a signal and identifies the arm-lifting gesture during card swiping using an ultrasonic transceiver device, thereby realizing double recognition of the identity of the smart card and the arm-lifting gesture.

[0006] The identity recognition of the arm-lifting gesture includes the following steps:

[0007] Step 1, continuously emit ultrasonic signals to the arm of the card swiping person by using the ultrasonic transceiving device arranged near the smart card reader when swiping the card, and continuously receive feedback signals until all ultrasonic feedback signals in the whole card swiping arm lifting gesture process are obtained, and the signals reflected by the arm in this process have 16 path reflections, so 16 channel time sequence data are received;

[0008] Step 2, calculate the energy spectrum density ESD information of the arm lifting action, pre-process the received ultrasonic reflection signals, and the received 16 channel data are specifically: acoustic tone T (T) = sin (2πft+b), t represents time, f represents frequency 48 KHz, b represents bias constant, T represents the time of the whole arm lifting gesture, and the 16 channel data are finally received at time T, and are represented as:

[0009]

[0010] Wherein Ω = 16 represents the set of all acoustic signal channels, the sampling frequency f = 48 KHz, A i is the amplitude coefficient of the acoustic signal of path i, the signals of the 16 channels correspond to a 16-dimensional matrix, the signals of the 16 channels are added to obtain a 1×N matrix, a set of ultrasonic data arranged in time is obtained, the ultrasonic data are detected and abnormal values are replaced, trend items are removed, and the ultrasonic data are jointly filtered by using Butterworth filtering and wavelet denoising, so that the ultrasonic data signal is more stable, and the signal-to-noise ratio is increased;

[0011] Step 3: extract the signal representing the arm lifting action in the ultrasonic data: select a periodic hamming window function to frame the ultrasonic data, calculate the power energy spectrum of the segmented signals respectively, use the hamming window function threshold method to distinguish the action and noise, and use the set empirical threshold value to judge which part is the action and which part is only noise through the power spectrum density ESD value;

[0012] Step 4: use the collected arm lifting gesture data as a training set to train the improved LSTM-FCN model, so that the trained improved LSTM-FCN model becomes a model capable of exclusively judging the training arm lifting action;

[0013] Step 5: input the arm lifting gesture data with judgment to the trained improved LSTM-FCN model, the trained improved LSTM-FCN model extracts the similar proportion of the user identity feature from the input information center, and then uses a small sample training support vector machine (SVM) to perform a classification task, and completes the voice signal-based arm lifting gesture identity authentication by judging the similarity.

[0014] Further, in step 2, a sliding window of N = 2k + 1 is set when detecting and replacing abnormal values, N is the window width, K is the sliding step, and a threshold A is set to distinguish abnormal values and normal values; when any sample point in the window deviates from the median value in the window by A standard deviations, the sample point is judged to be an abnormal point, and the median value is used to replace the abnormal point; set K = 3, A = 3; for the detrending term, the energy spectral density ESD information of the arm-raising gesture is expressed as: Y(t) = a + bt + e(t), a represents the intercept of the data, b represents the slope of the data, and e(t) represents the random error term in the data;

[0015] The arm-raising gesture signal is filtered by high-pass filtering and wavelet denoising, the order of the high-pass filter is 2, and the cutoff frequency is 15 Hz; wavelet denoising, wavelet coefficients exceeding the threshold are considered to be generated by the arm-raising gesture signal and are retained; on the contrary, the coefficients below the threshold are considered to be generated by noise and are set to zero for denoising. The threshold is determined according to the signal-to-noise ratio of the original signal.

[0016] Further, the improved LSTM-FCN model includes a long short-term memory network LSTM and a fully convolutional neural network FCN connected in parallel with the input, the output of the long short-term memory network LSTM and the output mean of the fully convolutional neural network FCN are mixed through a global pooling Concat function with a channel number of 256 and then output to a support vector machine SVM; wherein the long short-term memory network LSTM includes LSTM-1 and LSTM-2 connected in series, and LSTM-1 and LSTM-2 respectively include 64 and 128 neurons for input and output; the fully convolutional neural network FCN includes Conv-1 and Conv-2 connected in series, Conv-1 is a convolution block with a 3x3 convolution, and the effective feature information of the signal is extracted through the neighborhood induction of the convolution, after the 3x3 convolution, a point-by-point convolution with a convolution kernel of 1x1 is connected, and a compression and excitation module (Squeeze-and-Excitation) is arranged after each convolution layer, the compression and excitation module includes two sub-modules of a compression module and an excitation module, a batch normalization layer (BatchNorm) is arranged after the compression and excitation module, and an activation function GELU is connected after the batch normalization layer;

[0017] A Dropout layer is arranged after LSTM-1 and LSTM-2, and the probability of the Dropout layer is set to 0.5, which means that each neuron has a 50% probability of being randomly discarded during the training process;

[0018] The concatenated vector of the FCN and LSTM outputs is input into a fully connected layer and then into a Softmax function for classification prediction. The Softmax function calculates the probability distribution of each class to determine the likelihood of the input data belonging to each class.

[0019] When training the model, according to the determined positive sample class label data, the time sequence information and convolutional features of the collected ultrasonic gesture signals are extracted by the LSTM and FCN networks respectively, and the feature vector is input into a layer of SVM classifier. The SVM constructs an optimal hyperplane in the feature space to divide the data points into different categories. The model trains the LSTM-FCN feature extractor by extracting the time sequence signal features irrelevant to the class, and then uses a small sample support vector machine (SVM) for classification tasks.

[0020] Further, the setting of the ultrasonic transceiver device is selected according to needs, including: the ultrasonic transceiver device is arranged near the card reader, so as to ensure that the recognition process is not blocked; the mobile phone is used as the ultrasonic transceiver device, only need to ensure that the mobile phone is placed near the arm lifting action, and the mobile phone speaker is used to emit sound waves, and the microphone is used to receive.

[0021] An ultrasonic arm lifting gesture authentication device, comprising: an ultrasonic generating device, an ultrasonic receiving device, a processor, and a memory; the ultrasonic generating device is used to emit ultrasonic waves to the arm lifting gesture of the nearby personnel when they perform card swiping; the ultrasonic receiving device is used to receive the feedback signal of the emitted ultrasonic waves; the memory is used to store program codes; the processor is used to call the program codes; and the above-mentioned method for two-factor identity authentication of arm lifting gesture based on acoustic signals is executed.

[0022] A computer readable storage medium, in which a computer program is stored, the computer program being adapted to be loaded by a processor and execute the above-mentioned method for two-factor identity authentication of arm lifting gesture based on acoustic signals.

[0023] Beneficial effects: the present application proposes a kind of based on acoustic signal's arm-lifting gesture two-factor identity authentication method and designs innovative signal detection algorithm for arm-lifting behavior, can effectively extract arm-lifting gesture signal on the basis of card recognition, utilize the identification of arm-lifting gesture, to judge whether the card person is the correct holder of identity card, realize double insurance on the basis of card identification.The present application extracts user characteristics by improved LSTM-FCN network, then combines SVM classifier to construct authentication model, by the way of fine-tuning of transfer learning, the pre-training model is expanded to different arm-lifting gesture recognition tasks.The method has made remarkable improvement in accuracy and robustness.Effectively solve the security risks existing in the prior art identity authentication, and improve the accuracy and stability of authentication by optimizing model. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 It is the schematic diagram of arm-lifting gesture acoustic signal acquisition in the embodiment of the present application.

[0025] Figure 2 It is the overall waveform graph of signal denoising effect comparison before and after in the embodiment of the present application.

[0026] Figure 3 It is the performance column chart of detection accuracy of the method of the present application through experiment comparison.

[0027] Figure 4 It is the ROC curve of the working characteristics of all user identity recognition in the embodiment of the present application.

[0028] Figure 5 It is the performance comparison graph of user authentication model constructed by the present application and existing network in the embodiment of the present application.

[0029] Figure 6 It is the security evaluation column chart of the present application in experiment through three attack situations. DETAILED DESCRIPTION

[0030] The embodiments of the present application are further described in conjunction with the drawings:

[0031] The present application discloses a kind of based on acoustic signal's arm-lifting gesture two-factor identity authentication method, by being set in the ultrasonic device near card swiping, the action of cardholder in the process of card swiping is collected and identified, to confirm whether the identity of cardholder is card holder again;The arm-lifting action of card swiping process is as follows:

[0032] Step 1: data acquisition. Acoustic signal reflected due to user arm-lifting action is acquired.

[0033] Step 2: Signal detection and extraction. First, the 16 channels are summed, and then the superimposed channel data is jointly denoised to obtain a 1xN matrix in the algorithm, and abnormal values are detected and replaced, trend items are removed, and a Butterworth filter and wavelet denoising joint filter are selected. After the above work is completed, the power energy spectrum of the segmented signal is calculated, the window function threshold method is used to distinguish the action and noise, and the action extraction is completed.

[0034] Step 3: The LSTM-FCN model is improved by combining the Squeeze-and-Excitation module and the excitation function GELU, the deep features of the signal are extracted using the model network structure, and then the support vector machine (SVM) is used for classification task to realize the identity authentication of the user.

[0035] The present application mainly solves the problems existing in the existing identity authentication method, provides a more covert, more close to real life scene, and more easy to promote two-factor identification method for card authentication scene, solves the problems of being easily affected by environmental factors and needing additional equipment in the existing identity recognition, and the method is to use the user's smart device to emit ultrasonic waves, and deploy an Android APP on the smart phone to obtain the sound signal reflected by the user's arm lifting action. After preprocessing the obtained sound signal, the arm lifting action signal is extracted. A high-performance and lightweight identity authentication model is constructed, and then the pre-trained model is applied to the arm lifting gesture recognition task of the sound signal through the way of transfer learning.

[0036] As shown in Figure 1 The specific method flow of the present application is as follows:

[0037] Input: arm lifting gesture action.

[0038] Output: identity recognition result

[0039] 1. In the data acquisition stage, the energy spectrum density (ESD) of the sound signal describes the distribution of the energy of the sound signal in space with frequency, and can perceive the motion of all objects around with the change of energy distribution, therefore the ESD of the sound signal obtained by the smart phone is used in the present application. It can be deployed as an Android APP on the phone, the smart phone's built-in speaker emits ultrasonic waves, and the sound signal reflected by the arm lifting action is received through the microphone.

[0040] 2. In the signal detection and extraction stage, the obtained 16 channels are first summed in the algorithm to obtain a 1xN matrix, and the signal segments with large noise amplitude at the beginning and end are removed. The hampel filtering method is used to detect and remove outliers of the collected time series data. The linear trend component in the data is obtained by finding the best fitting straight line of the data, and it is subtracted from the data to remove the trend term. Assume that there is a set of time series data Y(t) with length N, where t represents the time step. For linear trend removal, each data point can be represented as:

[0041] Y(t) = a + bt + e(t)

[0042] By minimizing the sum of squares of errors, the solution of the best fitting straight line is obtained:

[0043]

[0044] Subtracting this linear trend component from the original data gives the detrended time series data. After the above processing, the data still includes environmental noise and interference signals generated by human body shaking. The invention uses a combination of high-pass filtering and wavelet denoising to smooth the signal. The generation formula of the Butterworth high-pass filter is

[0045]

[0046] where n is the order of the filter, and D0 is the cutoff frequency of the Butterworth high-pass filter. When wavelet denoising, let the ideal original signal be

[0047] d(t) = f(t) + σz(t)

[0048] where f(t) is the signal not contaminated by noise, z(t) is the noise, and σ is the noise intensity. The signal d(t) is decomposed by wavelet to obtain a set of wavelet decomposition coefficients Wj. The wavelet coefficients Wj are threshold processed, and the signal is reconstructed by inverse wavelet transform to obtain the output signal f'(t), which is the denoised signal. Joint filtering aims to minimize the interference caused by noise while preserving important motion information, making it easier to segment the signal. After completing the data preprocessing, the invention selects a periodic hamming window for framing, calculates the power spectrum of the segmented signal, and judges the part where the motion occurs according to the difference in power spectrum density, as shown in Figure 2 The schematic diagram before and after denoising is shown.

[0049] In the identity authentication model building stage, based on the obtained information, combining the advantages of deep learning and traditional machine learning, a pre-training-fine-tuning paradigm model is established. Through the use of an improved LSTM-FCN feature extractor, a small sample support vector machine (SVM) is used for classification tasks.

[0050] In the feature extraction period, an improved LSTM-FCN network is used. In the FCN, two convolutional blocks are used for stacking, the first block consists of 64 feature output channels, and the second block has 128 output channels. The initial block is a 3x3 convolution, in order to optimize the performance and fusion characteristics of the model, a 1x1 point convolution is used on the basis of the original network to reduce the number of parameters. After learning the weights of different convolution kernels, the network adjusts and combines the features from different channels to enhance the representation ability of the model. In addition, a squeeze and excitation module is added to each convolutional layer, in the squeeze module, a global average pooling operation is used to compress the feature map of each channel to capture the global feeling field of the channel. In the excitation module, a small fully connected network including two fully connected layers is used, the feature vector obtained in the last step is used as input to learn the excitation weights of the channel. These weights can adaptively adjust the importance of each channel to enhance the representation of different features. After excitation, GELU is selected as the activation function of the post-batch normalization layer to normalize and standardize the input, accelerating the training process of the model. In addition, two LSTM blocks are used and combined with Dropout layers to effectively learn and capture the time information in the input sequence. In the user authentication and transfer learning period, the previously obtained knowledge and model are transferred to the new learning task using the fine-tuning-based transfer learning, first using the LSTM-FCN feature extractor to extract the class-independent time signal features, then using a small sample radial basis function kernel to train the SVM for authentication, and finally used to solve the new learning task.

[0051] As shown in Figure 2 , the experimental results show that the proposed data processing method can effectively reduce the noise in the signal. As shown in Figure 3 , the experimental results show that the proposed arm-lifting gesture authentication method based on acoustic signals has an average false negative rate (FNR) of 4.973%, an average false positive rate (FPR) of 5.936%, and an average detection accuracy (accuracy) of 97.191%, with very high accuracy. From Figure 4 , Figure 5 , it can be seen that the proposed authentication model can perform well in user identification and authentication. Compared with the existing LSTM-FCN network, the proposed model improves the authentication accuracy by 2.4%, reduces the FAR and FRR values, and significantly reduces the number of parameters. As shown in Figure 6As shown, the detection accuracy of the violent attack is 97.414%, the detection accuracy of the mimic attack is 97.170%, and the detection accuracy of the replay attack is 97.273%, which shows high security.

[0052] Therefore, it has the following advantages: the detection behavior is more concealed, the detection action is close to the real life scene, no additional equipment is needed, and it is easier to promote. By using the LSTM-FCN network to first construct a deep learning model for pre-training, and then using an SVM classifier for classification, the accuracy and robustness are improved, so that the proposed identity authentication method has higher security.

Claims

1. A method for two-factor authentication of a raise-arm gesture based on an acoustic signal, characterized in that, While identifying the hand-held smart card through the signal, the ultrasonic transceiving device is used to identify the arm-lifting gesture during the card swiping, so as to realize the double identification of the smart card and the arm-lifting gesture identity; The identity recognition of the arm-lifting gesture comprises the following steps: Step 1, the ultrasonic transceiving device arranged near the smart card reader continuously emits ultrasonic signals to the arm during the card swiping and continuously receives the feedback signals until all the ultrasonic feedback signals in the whole arm-lifting gesture during the card swiping are acquired, and there are 16 path reflections of the signals reflected by the arm during the process, so 16 channel time sequence data are received; Step 2, the energy spectrum density ESD information of the arm-lifting action is calculated, the received ultrasonic reflection signals are preprocessed, and the 16 channel data are specifically: acoustic tone T(T) = sin(2πft+b), t represents the time, f represents the frequency 48KHz, b represents the bias constant, T represents the time of the whole arm-lifting gesture, and the 16 channel data are finally received at the time T, and are represented as: where Ω = 16 represents the set of all acoustic signal channels, the sampling frequency f = 48 KHz, A i is the amplitude coefficient of the acoustic signal of path i, the 16-channel signal corresponds to a 16-dimensional matrix, the 16-channel signal is added to obtain a 1xN matrix, a set of ultrasonic data arranged in time is obtained, the ultrasonic data is detected and abnormal values are replaced, a trend item is removed, and a Butterworth filter and a wavelet are selected to jointly filter the ultrasonic data, so that the ultrasonic data signal is smoother and the signal-to-noise ratio is increased; Step 3: the signal representing the arm-lifting action in the ultrasonic data is extracted: a periodic hamming window function is selected to frame the ultrasonic data, the power energy spectrum of the segmented signals is calculated respectively, the hamming window function threshold method is used to distinguish the action and the noise, and the experience threshold is set to judge which part is the action and which part is only noise through the power spectrum density ESD value; Step 4: the collected arm-lifting gesture data are used as a training set to train the improved LSTM-FCN model, so that the trained improved LSTM-FCN model becomes a model capable of exclusively judging the training arm-lifting action; Step 5: the arm-lifting gesture data with judgment are input to the trained improved LSTM-FCN model, the trained improved LSTM-FCN model extracts the similar proportion of the user identity feature from the input information center, and then uses a small sample training support vector machine (SVM) to perform a classification task, so as to complete the arm-lifting gesture identity authentication based on the acoustic signal by judging the similarity.

2. The voice signal based arm-raising gesture two-factor authentication method of claim 1, wherein: In step 2, when detecting and replacing the abnormal value, a sliding window of N = 2k+1 is set, N is the window width, k is the sliding step, a threshold A is set to distinguish the abnormal value and the normal value; if any sample point in the window is different from the median value in the window by A standard deviations, the sample point is judged as an abnormal point, and the median value is used to replace the abnormal point; k = 3 and A = 3 are set, and for the detrending term, each data point of the energy spectrum density ESD information of the arm-lifting action is represented as: Y(t) = a + bt + e(t), a represents the intercept of the data, b represents the slope of the data, and e(t) represents the random error term in the data; The lifting arm gesture signal is filtered jointly by high-pass filtering and wavelet denoising, the order of the high-pass filter is 2, and the cut-off frequency is 15Hz; wavelet denoising, wavelet coefficients exceeding the threshold are considered to be generated by the lifting arm gesture signal and are retained; on the contrary, the coefficients below the threshold are considered to be generated by noise and are set to zero for denoising; the threshold is determined according to the signal-to-noise ratio of the original signal. 3.The method of claim 1, wherein the method further comprises: receiving a voice signal from the user; and determining whether the voice signal matches a voice signal stored in the database. The improved LSTM-FCN model comprises a long short-term memory network LSTM and a fully convolutional neural network FCN connected in parallel with the input, and the output of the long short-term memory network LSTM and the output mean of the fully convolutional neural network FCN are mixed through a global pooling Concat function with a channel number of 256 and then output to a support vector machine SVM; wherein the long short-term memory network LSTM comprises LSTM-1 and LSTM-2 connected in series, and LSTM-1 and LSTM-2 respectively contain 64 and 128 neurons for input and output; the fully convolutional neural network FCN comprises Conv-1 and Conv-2 connected in series, Conv-1 is a convolution block with a 3*3 convolution, and the effective feature information of the signal is extracted through the neighborhood induction of the convolution, after the 3*3 convolution, a point-by-point convolution with a convolution kernel of 1*1 is connected, and a Squeeze-and-Excitation compression and excitation module is arranged after each convolution layer, the Squeeze-and-Excitation compression and excitation module comprises two sub-modules of a compression module and an excitation module, a Batch Norm layer is arranged after the Squeeze-and-Excitation compression and excitation module, and an activation function GELU is connected after the Batch Norm layer; A Dropout layer is arranged after LSTM-1 and LSTM-2, and the probability of the Dropout layer is set to 0.5, which means that each neuron has a 50% probability of being randomly discarded during the training process; During training, the vector obtained by concatenating the results output by FCN and LSTM is input into a Softmax function through a fully connected layer for classification prediction, and the Softmax function calculates the probability distribution of each class to determine the possibility of the input data belonging to each class; When training the model, according to the determined positive sample class label data, the time sequence information and convolution features of the collected ultrasonic gesture signal are extracted through the LSTM and FCN networks respectively, and the feature vector is input to a layer of SVM classifier; the SVM constructs an optimal hyperplane in the feature space to divide the data points into different categories; the model trains the LSTM-FCN feature extractor by extracting the time sequence signal features irrelevant to the class, and then uses a small sample to train the support vector machine (SVM) for classification tasks.

4. The voice signal based arm-up gesture two-factor authentication method of claim 1, wherein: The arrangement of the ultrasonic transceiver device is selected according to needs, including: the ultrasonic transceiver device is arranged near the card swiping receiver, so as to ensure that the identification process is not blocked; the mobile phone is used as the ultrasonic transceiver device, and it is only needed to ensure that the mobile phone is placed near the arm lifting action, the mobile phone loudspeaker is used to emit sound waves, and the microphone is used to receive.

5. An ultrasonic arm-lifting motion authentication device characterized by comprising: Comprise: An ultrasonic wave generating device, an ultrasonic wave receiving device, a processor, and a memory; The ultrasonic wave generating device is used to emit ultrasonic waves to the arm lifting gesture of the person near the card swiping; the ultrasonic wave receiving device is used to receive the feedback signal of the emitted ultrasonic waves; the memory is used to store program codes; and the processor is used to call the program codes; To execute the sound signal based arm lifting gesture two-factor identity authentication method of any one of claims 1-3.

6. A computer readable storage medium characterized by, The computer readable storage medium stores a computer program, and the computer program is suitable for being loaded by the processor and executing the sound signal based arm lifting gesture two-factor identity authentication method of any one of claims 1-3.

Images