Smart phone security protection system and method based on Internet

Abstract

The invention discloses a smart phone security protection system and method based on the Internet, and the method comprises the steps: collecting behavior data of a touch click operation of a user on a smart phone, and carrying out the preprocessing of the behavior data; analyzing and extracting a feature set of a click behavior mode by taking the preprocessed acceleration, a single vibration signal captured by a gyroscope sensor and touch data of a touch screen as input, and performing feature selection on the extracted feature set to represent a key feature subset of user behavior characteristics; and training a behavior model corresponding to the behavior data by adopting a single classifier for anomaly detection according to the key feature subset, authenticating the legality of the behavior, obtaining the gait data of the to-be-authenticated user, and setting an authentication mechanism based on the modal data of the touch screen sensor and the gait data to realize the security protection of the smart phone. The problem of incomplete oscillation signals caused by low sampling rate of the sensor is solved, the user behavior model is trained by adopting a single classification algorithm, and the balance between the safety and the availability of the authentication method can be improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of mobile phone security protection, and in particular relates to an Internet-based smart phone security protection system and method. Background Art

[0002] With the development of mobile Internet, smartphones have gradually transformed from communication devices to more diversified social and entertainment tools. More and more applications are deployed on mobile phones, providing many conveniences for our lives. However, in order to provide accurate and personalized services, some mobile applications (such as WeChat, Alipay, etc.) collect a large amount of personal sensitive data of users, including geographic location, photos and payment details, etc., which poses the risk of privacy leakage and information theft. At the same time, during the use of the device, a large amount of sensitive data closely related to user privacy, such as chat records and financial data, is generated and stored, which leads to a high degree of privatization of mobile phones, and the security issues of application operations and privacy data on them are also receiving more and more attention.

[0003] Smartphones are mainly threatened by illegal control and need to adopt identity authentication mechanisms to protect privacy and security. Knowledge-based and biometric-based identity authentication is a commonly integrated authentication method in the industry. Knowledge-based identity authentication relies on user memory, such as passwords, unlock patterns, etc., but knowledge-based authentication is troublesome to remember and is vulnerable to malicious attacks or software threats, such as password exhaustive, guessing attacks, and malicious touch recording programs. Biometric-based identity authentication uses the user's own unique physiological characteristics as the basis for identification. There is no problem of credentials carrying or memory trouble, and the identification is accurate and fast. Commonly used biometric information includes face, fingerprint or voice, but such information is easily interfered with. For example, fingerprint stains, facial expressions and changes in light in the external environment will affect the recognition effect, and there are problems such as theft, copying and forgery attacks. Both authentication methods have extremely high accuracy and stability, and can efficiently authenticate the user's identity once at the entrance of the smartphone system or application, but there are deficiencies in the legitimacy verification and privacy security protection of the user after the device is unlocked. Summary of the invention

[0004] In view of this, the present invention provides an Internet-based smartphone security protection system and method, which solves the problem of imperfect oscillation signals caused by low sensor sampling rate, improves the balance between security and usability of the authentication method, and is specifically implemented by the following technical solutions.

[0005] In a first aspect, the present invention provides an Internet-based smart phone security protection system, comprising:

[0006] A data collection module is used to collect the behavior data of the user's touch and click operations on the smart phone and pre-process the behavior data, wherein the behavior data includes motion sensor modal data and touch screen sensor modal data;

[0007] A feature extraction module is used to analyze and extract a feature set of a click behavior pattern using the preprocessed acceleration, a single vibration signal captured by a gyroscope sensor, and touch data of the touch screen as input, wherein the click behavior pattern includes a force characteristic and a speed characteristic;

[0008] A feature selection module is used to select the extracted feature set to obtain a key feature subset that characterizes the user behavior characteristics;

[0009] A behavior authentication module is used to train a behavior model corresponding to the behavior data using a single classifier for anomaly detection based on a subset of key features, and authenticate the legality of the behavior, wherein the behavior model includes legal behavior data and illegal behavior data;

[0010] Identity decision evaluation is used to obtain the gait data of the user to be authenticated, and to set up an authentication mechanism based on the touch screen sensor modal data and gait data to achieve security protection for smartphones.

[0011] As a preferred embodiment of the above technical solution, collecting the behavior data of the touch and click operation on the smart phone and preprocessing the behavior data include:

[0012] Each sampling point MS of the motion sensor modal data i Including X, Y, Z three-dimensional data of accelerometer acc and gyroscope gyro, sampling timestamp is TmSp, touch occurrence flag TaFg, the corresponding sampling point expression is:

[0013] MS i =[accX i ,accY i ,accZ i ,gyroX i ,gyroY i ,gyroZ i ,TmSp i ,TaF i ], the touch screen modal data of a single touch click includes two touch events at the time of pressing and lifting, recorded as TM = [TE down ,TE up], the single touch event data structure TE is expressed as TE = [posX, posY, press, major, minor, oritn, EvTe], with the upper left corner of the touch screen as the origin, posX and posY as the horizontal and vertical coordinates of the touch point, press as the pressure applied by the finger, major and minor as the major and minor axes of the elliptical contact surface between the finger and the touch screen, oritn as the direction of the contact surface, EvTe indicates the occurrence of a touch event, and the motion sensor modal data and the touch screen sensor modal data are associated and synchronized according to their respective recorded timestamps to achieve a one-to-one correspondence between the motion sensor modality and the touch screen modality of a single click;

[0014] When a touch click operation occurs, the touch occurrence flag TaFg of the motion sensor sampling point generates a continuous "1" to identify this signal sequence as a click vibration sequence, and the click vibration signal is preprocessed, extracted and segmented to generate a user behavior data set, where data preprocessing includes abnormal data cleaning, click vibration processing and modal data association.

[0015] As a preferred embodiment of the above technical solution, the pre-processed acceleration, the single vibration signal captured by the gyroscope sensor and the touch screen touch data are used as input for analysis and extraction of the feature set of the click behavior pattern, including:

[0016] Extract behavioral characteristics that can reflect behavioral data, including the energy, duration, and click vibration characteristics corresponding to the time domain changes of the vibration signal captured by the motion sensor, as well as the position, pressure, contact area, and touch duration of the touch screen data;

[0017] A preset n-point vibration signal sequence is S = {s0, s1, ..., s n-1}, S can be a complete click vibration sequence or a sub-stage pressing sequence and a recovery sequence. The types of click vibration features extracted from the vibration signal sequence include time domain distribution features, energy-related features, and time-related features. Among them, the time domain distribution features are used to reflect the time domain statistical distribution characteristics of the vibration signal; the energy-related features are directly related to the strength characteristics of the user's click behavior. The energy-related features include the signal short-time energy STE and the power spectrum density energy PSD. The short-time energy reflects the time domain energy characteristics, and the calculation expression is: PSD is the distribution of power with frequency, which is used to reflect the energy characteristics in the frequency domain. The corresponding expression is: Where fs represents the sampling rate, and FFT() represents the fast Fourier transform; the time-related features are used to reflect the speed characteristics of the click behavior, the click behavior and the duration of different stages of the click;

[0018] According to each dimension of the motion sensor data and its two subsequences, ten vibration features are extracted from each subsequence, and six feature types are extracted from the complete sequence. Therefore, each set of motion sensor data can extract 2*10+6=26 dimensional features, and the X, Y, and Z three-dimensional data of the motion sensor can extract a total of 3*26=78 dimensions;

[0019] The energy amplitude sequence AMP of the motion sensor is not segmented, and ten features are directly extracted from it. 78+10=88-dimensional features can be extracted from one type of sensor data, and the total dimension of click vibration features extracted from the motion sensor modal data is 88*2=176.

[0020] As a preferred embodiment of the above technical solution, the extracted feature set is subjected to feature selection to characterize a key feature subset of user behavior characteristics, including:

[0021] The Fisher score algorithm is used to measure the distinguishability of features based on the fact that the distance between key features is small within the class and large between classes, including the distance between classes S B and intra-class distance S W The corresponding expressions of the two metrics are: Among them, S B (f k ), S w (f k ) represents each dimension feature f k Calculate the corresponding evaluation index, U represents the sample category set, n i represents the i-th class U i The number of samples, the fth k Wei Jiwei For U i Sample at f k The mean value on i is the proportion of samples of the i-th category in all samples, m k For all samples in the fth k The mean value over the dimension;

[0022] Assign a score J to each dimension feature Fisher (f k )=S B (f k ) / S w (f k ), S B With S w The ratio of is used to measure feature importance.

[0023] As a preferred embodiment of the above technical solution, the behavior authentication module includes:

[0024] Data preprocessing layer: When the user touches the screen, the sensor data is recorded and the raw data is obtained. The mean compensation algorithm is used to fill the missing values ​​of the collected raw data and filter out the abnormal values. Then, the discrete cosine transform is used to transform the raw data into the frequency domain to expand the feature space. Then, the noise components with too high and too low frequencies are filtered out to obtain a set of third-order tensors. The three dimensions of the third-order tensor are the sensor dimension, the sampling frequency of the original signal, and the frequency phase of the frequency domain signal.

[0025] Convolution layer: It includes an independent convolution layer corresponding to the third-order tensor of each sensor, a merged convolution layer that merges the independent convolution outputs of each sensor tensor, and a merged convolution layer that merges the independent convolution outputs of each sensor tensor. A multi-layer independent convolution layer structure is set to train and learn the internal relationship and extract the feature adjacent matrix. The matrix is ​​then converted into a one-dimensional adjacent matrix by splicing and passed to the merged convolution layer as input;

[0026] Recurrent layer: Input the one-dimensional feature vector output by the previous layer into a two-layer recurrent neural network and make an identity judgment for each user touch behavior;

[0027] Output layer: removes a number of judgment results obtained from the previous layer through a sliding window, and then inputs them into a fully connected neural network to give the final verification result.

[0028] As a preferred embodiment of the above technical solution, the original data becomes a set of d after data preprocessing. (k) *2f*N three-dimensional tensor, where d (k) represents the dimension of the kth sensor, f represents the number of frequency-phase pairs in the frequency domain, and N represents the number of input data matrices. represents the processed data matrix of the kth sensor in the nth touch cycle;

[0029] The data matrix As the input of an independent convolutional layer and perform feature extraction on multiple sensor data groups, the input matrix There is a relationship between frequency domain features and sensor dimension features that needs to be extracted, using a d (k) *c1's 2D convolution kernel learning matrix The correlation between the sensor measurement dimension and the local modes in the frequency domain is obtained, and the output matrix Enter the next convolutional layer;

[0030] According to the matrix Use two one-dimensional convolution kernels of 1*c2 and 1*c3 respectively to learn the abstract and high-level relationship between features, and get the output matrix and

[0031] The k-dimensional feature vector matrix generated by the features of multiple sensors is obtained through independent convolutional layers The eigenvector matrix The merged convolution layer is used as input, and the integrated feature vector matrix is ​​used for feature extraction;

[0032] Use a K*c4 two-dimensional convolution kernel to learn the intrinsic connection between K sensor feature vectors to get an output Use a one-dimensional 1*c5 convolution kernel and a one-dimensional 1*c6 convolution kernel to calculate and get and Among them, each convolution layer has a corresponding normalization operation to constrain the data to a fixed distribution range to avoid gradient vanishing and gradient explosion, and the expression of using ReLU as the activation function in each convolution layer is: After a concatenation operation, a one-dimensional feature vector x is obtained as the output of this convolutional layer.

[0033] As a preferred embodiment of the above technical solution, the mean square error is used as the loss function, and the Adam algorithm is used in the Droupout layer to optimize the convergence speed. The corresponding expression is: Among them, m t represents the exponential decay bias of the first-order moment estimate, v t represents the exponential decay bias of the second-order moment estimate. β1 and β2 are the recommended default values ​​of the decay coefficients and are 0.9 and 0.999, respectively. t represents the gradient;

[0034] After initialization, the error is corrected and the corrected m is obtained by calculation. t and v t To offset the deviation, the corresponding expression is Then the final gradient update rule is expressed as Where α represents the learning rate and ε represents the deviation compensation. The recommended default value is 10 -8 ;

[0035] Each touch will generate a feature vector x after processing. It is assumed that there are N touches in the whole verification process to obtain a vector set X = {x1, x2...x N}, and input them into the proposed network structure in sequence to eventually generate the corresponding touch recognition results.

[0036] As a preferred embodiment of the above technical solution, the accelerometer triaxial data is processed, and the amplitude M is used as the criterion for gait cycle segmentation. The expression of M is: The process of period segmentation based on the M value includes the following specific steps:

[0037] The local minimum value of M is calculated and used as the candidate set of the starting point of the gait cycle;

[0038] Calculate the average of all local minima and remove local minima above the average from the candidate set;

[0039] The candidate points that are adjacent in time are grouped together, and the minimum value in each group is selected as the starting point of the period segmentation. After the period segmentation is completed according to the M value, the period division time of the nine dimensions of the motion sensor measurement value is consistent with the starting point of M;

[0040] The accelerometer data is calibrated, and then the three-axis directions of the new coordinate system are obtained based on the calibrated data. η represents the X, Y, and Z axes of the user coordinate system respectively. The η axis of the coordinate system with the user as the reference is parallel to the user's torso. The axis is the front-to-back direction, and its positive direction is consistent with the user's movement direction. The γ axis is the left-to-right direction. The axis is perpendicular to the η axis,

[0041] Includes calibration of accelerometer data to user coordinate system γ, Generation of the η axis, the accelerometer data collected in the device coordinate system Ac′=[Ac′ X ,Ac′ Y ,Ac′ Z ] T Expressed as Where, Ac=[Ac x ,Ac Y ,Ac Z ] T is the calibrated accelerometer vector, K is the sensitivity matrix; B = [b1, b2, b3] T is the deviation, n=[n1,n2,n3] T is a random noise close to 0;

[0042] By constructing the objective function as Where m represents the number of measurement locations, and g represents the local gravitational acceleration;

[0043] The gravity component contained in the sensor data collected by the device coordinate system is not fixed. The average gravity direction is used as the η axis of the user coordinate system. It is assumed that the current gait cycle is t (t = 1, 2...). In the gait cycle t, the gravity component g s It can be expressed as in, They represent the average values ​​of the three axes of the accelerometer in the gait cycle t respectively; the expression of the η axis in the user coordinate system is: ‖·‖ represents the modulus operation of the vector, and the accelerometer vector Ac = [AcX ,Ac Y ,Ac Z ] T , gyroscope vector Gr = [Gr X ,Gr Y ,Gr Z ] T And the magnetometer vector Ma = [Ma X ,Ma T ,Ma Z ] T The projections on the η axis of the user coordinate system are Ac η , Gr η and Ma η , the corresponding calculation expression is

[0044] As a preferred embodiment of the above technical solution, the gravity component of the user coordinate system can be obtained to obtain its component on the horizontal plane. The positive direction of the axis is based on the PCA principle to find the direction with the largest variance in the original data. The direction is parallel to the user's movement direction, and the user coordinate system can be obtained. Axis, the specific algorithm steps include:

[0045] Removing the gravity component of the accelerometer, we get

[0046] Ac h The average value of X, Y, and Z axis data

[0047] Ac h The three-axis data is decentralized, and the average value corresponding to each axis is subtracted from the value of each axis. Ac h The covariance matrix of Where n represents the number of samples in a gait cycle;

[0048] Get the covariance matrix Cov(Ac h ), the eigenvalue and eigenvector of the maximum eigenvalue, and the eigenvector v corresponding to the maximum eigenvalue, v represents the eigenvalue in the user coordinate system. The direction corresponding to the axis is expressed as Accelerometer vector Ac = [Ac X ,Ac Y ,Ac Z ] T In the user coordinate system The projection onto the axis can be expressed as

[0049] The corresponding gyroscope vector Gr = [Gr X ,Gr Y ,Gr Z ] T and magnetometer vector Ma = [Ma X ,Ma Y ,Ma Z ] T It can be expressed as

[0050] From the characteristics of the coordinate system, we know that the γ axis is The γ axis of the user coordinate system is obtained by calculating the cross product of the first two vectors: Then the accelerometer vector Ac=[Ac X ,Ac Y ,Ac Z ] T , gyroscope vector Gr = [Gr X ,Gr Y ,Gr Z ] T And the magnetometer vector Ma = [Ma X ,Ma Y ,Ma Z ] T The projection on the γ axis of the user coordinate system can be expressed as Then a sensor measurement value {Ac X ,Ac Y ,Ac Z ,Gr X ,Gr Y ,Gr Z ,Ma X ,Ma Y ,Ma Z} is eventually converted to

[0051] In a second aspect, the present invention further provides a smart phone security protection method based on the Internet, comprising the following steps:

[0052] Collecting behavioral data of the user's touch and click operations on the smartphone and preprocessing the behavioral data, wherein the behavioral data includes motion sensor modal data and touch screen sensor modal data;

[0053] The pre-processed acceleration, the single vibration signal captured by the gyroscope sensor and the touch screen touch data are used as input to analyze and extract the feature set of the click behavior pattern, wherein the click behavior pattern includes force characteristics and speed characteristics;

[0054] The extracted feature set is subjected to feature selection to obtain a key feature subset that characterizes user behavior characteristics;

[0055] A single classifier for anomaly detection is used to train a behavior model corresponding to the behavior data according to a subset of key features, and the legality of the behavior is authenticated, wherein the behavior model includes legal behavior data and illegal behavior data;

[0056] The gait data of the user to be authenticated is obtained, and an authentication mechanism is set based on the touch screen sensor modal data and gait data to achieve security protection of the smartphone.

[0057] The present invention provides an Internet-based smart phone security protection system and method. The system collects user behavior data of touch and click operations on a smart phone, pre-processes the behavior data, uses the pre-processed acceleration, a single vibration signal captured by a gyroscope sensor, and touch data of a touch screen as input for analysis and extracts a feature set of a click behavior pattern, performs feature selection on the extracted feature set to characterize a key feature subset of user behavior characteristics, trains a behavior model corresponding to the behavior data using a single classifier for anomaly detection based on the key feature subset, authenticates the legitimacy of the behavior, obtains gait data of a user to be authenticated, and sets an authentication mechanism based on touch screen sensor modal data and gait data to achieve security protection for the smart phone. The system solves the problem of imperfect oscillation signals caused by a low sensor sampling rate, eliminates the influence of redundant and irrelevant features on model accuracy, and uses a single classification algorithm to train the user behavior model, thereby improving the balance between security and usability of the authentication method. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0059] Figure 1 A structural block diagram of the Internet-based smart phone security protection system provided by the present invention;

[0060] Figure 2 A flowchart of the Internet-based smart phone security protection method provided by the present invention. DETAILED DESCRIPTION

[0061] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0062] In a first aspect, the present invention provides an Internet-based smart phone security protection system, comprising:

[0063] A data collection module is used to collect the behavior data of the user's touch and click operations on the smart phone and pre-process the behavior data, wherein the behavior data includes motion sensor modal data and touch screen sensor modal data;

[0064] A feature extraction module is used to analyze and extract a feature set of a click behavior pattern using the preprocessed acceleration, a single vibration signal captured by a gyroscope sensor, and touch data of the touch screen as input, wherein the click behavior pattern includes a force characteristic and a speed characteristic;

[0065] A feature selection module is used to select the extracted feature set to obtain a key feature subset that characterizes the user behavior characteristics;

[0066] A behavior authentication module is used to train a behavior model corresponding to the behavior data using a single classifier for anomaly detection based on a subset of key features, and authenticate the legality of the behavior, wherein the behavior model includes legal behavior data and illegal behavior data;

[0067] Identity decision evaluation is used to obtain the gait data of the user to be authenticated, and to set up an authentication mechanism based on the touch screen sensor modal data and gait data to achieve security protection for smartphones.

[0068] In this embodiment, the behavior data of the touch and click operation of the smart phone is collected and pre-processed, including: each sampling point MS of the motion sensor modal data i Including X, Y, Z three-dimensional data of accelerometer acc and gyroscope gyro, sampling timestamp is TmSp, touch occurrence flag TaFg, the corresponding sampling point expression is: MS i =[accX i ,accY i ,accZ i ,gyroX i ,gyroY i ,gyroZ i ,TmSp i ,TaF i ], the touch screen modal data of a single touch click includes two touch events at the time of pressing and lifting, recorded as TM = [TE down ,TE up], the single touch event data structure TE is expressed as TE = [posX, posY, press, major, minor, oritn, EvTe], with the upper left corner of the touch screen as the origin, posX and posY as the horizontal and vertical coordinates of the touch point, press as the pressure applied by the finger, major and minor as the major and minor axes of the elliptical contact surface between the finger and the touch screen, oritn as the direction of the contact surface, EvTe indicates the occurrence of a touch event, and the motion sensor modal data and the touch screen sensor modal data are associated and synchronized according to their respective recorded timestamps to achieve a one-to-one correspondence between the motion sensor modality and the touch screen modality of a single click; when a touch click operation occurs, the touch occurrence flag TaFg of the motion sensor sampling point generates a continuous "1" to indicate that this signal sequence is a click vibration sequence, and the click vibration signal is preprocessed, extracted and segmented to generate a click operation to generate a user behavior data set, wherein data preprocessing includes abnormal data cleaning, click vibration processing and modal data association.

[0069] It should be noted that financial and banking applications on smartphones often use payment scenarios with one-time random verification codes. In this case, users do not use payment passwords for transactions. Instead, the system sends a one-time random 6-digit verification code via SMS in real time when a transaction request is initiated. The numeric keypad for entering the verification code uses a random layout to enhance the dynamic security protection of transaction verification, but the random verification code inevitably has the security risk of being stolen and monitored. Based on the characteristics of such scenarios, the numeric keypad records the random numbers generated by the user input during the test process, generating exit click behavior data, including touch screen click scenarios. However, general keystroke behavior authentication does not focus on the behavioral characteristics of a single touch point, but rather uses keystroke characteristics such as keystroke time interval and frequency as identity information for user authentication. This results in keystroke behavior authentication being strongly dependent on the user's text input habits and having a high correlation with the user's familiarity with the keyboard layout and text. Therefore, the layout randomness and content randomness of the random numeric keypad are used to make users focus more on the click input of a single number, weakening the influence of the user's keystroke habits, so as to enhance the effectiveness of user identity authentication and privacy security.

[0070] The original data recorded by the touch screen may contain non-click behavior data, such as touch screen sliding or multi-touch operation data, while the click vibration signal captured by the motion sensor is a continuous long signal sequence, and the word click vibration signal needs to be extracted and segmented. In order to fully characterize the user's click behavior strength and speed behavior characteristics, the relevant behavior features are analyzed and extracted from the two modal data of motion to sensor and touch screen as comprehensively as possible. However, the feature set obtained in this way inevitably has a large dimension and is interfered by irrelevant or redundant features, which brings challenges to the subsequent classifier model training and certification. Feature selection is the process of selecting a feature subset that can meet the optimal evaluation criteria from the original feature set. Its purpose is to eliminate irrelevant and redundant features and screen out the best key feature combination, thereby achieving dimensionality reduction, improving model accuracy, and reducing running time. After completing the feature selection analysis for each user, the most representative feature set can be used to characterize their behavior characteristics. These relevant features can be calculated during authentication, which can greatly reduce the computational complexity and improve the authentication speed.

[0071] The touch free click behavior authentication is regarded as a single classification problem. In the training phase, only the click behavior data of the legitimate users of the device are used to train the user behavior model. In the authentication phase, the model is used to detect illegal behaviors, thereby converting identity authentication into an anomaly detection task, that is, detecting abnormal user operations in the authentication phase. Identity decision evaluation is that the algorithm maintains a fixed queue as the user behavior window. This queue and the user behavior authentication model take the current touch click behavior authentication results as input, and provide the continuous illegal behavior threshold N and the illegal behavior ratio threshold ε for the user identity illegality decision. When the current user click behavior authentication is illegal, the algorithm combines two strategies to decide the legitimacy of the user identity; when the behavior window queue is not full, whether the most recent N operations in the behavior window are continuously illegal is used as the decision strategy. If so, the user identity is illegal; when the behavior window queue is full, the user identity is determined by whether the proportion of illegal operation behaviors in the window is greater than the preset threshold ε. If it exceeds the preset threshold, the user identity is illegal. At this time, the system locks the device and clears the window. The user unlocks the device and accumulates the behavior window again. The thresholds N and ε determine the tolerance scale of illegal behaviors within the window. The larger the value, the higher the acceptable degree of misidentification of legitimate user behaviors, but the more likely it is to mistakenly allow illegal behaviors. The values ​​of the two are a measure of practicality and security. Therefore, the authentication method can more completely implement the legitimacy assessment of user identities, limiting illegal user operations as much as possible while taking practicality into account. In addition, since the reliability of identity assessment results depends on the number of click behaviors involved in an identity authentication decision, that is, the size of the behavior window, the window size setting introduces the balance problem of reliability and security. The larger the window, the higher the reliability of the legitimacy decision result of a user identity authentication. However, this also means that when the decision window is not full, the longer the illegal user can access, the larger the operation space, resulting in reduced system security.

[0072] It should be understood that by collecting the behavioral data of the user's touch and click operations on the smartphone and preprocessing the behavioral data, the preprocessed acceleration, gyroscope sensor captures a single vibration signal and touch screen touch data as input for analysis and extracting the feature set of the click behavior pattern, the extracted feature set is subjected to feature selection to characterize the key feature subset of the user's behavior characteristics, a single classifier for anomaly detection is used to train the behavior model corresponding to the behavior data based on the key feature subset, and the legitimacy of the behavior is authenticated, the gait data of the user to be authenticated is obtained, and an authentication mechanism is set based on the touch screen sensor modal data and gait data to achieve security protection of the smartphone, which solves the problem of imperfect oscillation signals caused by low sensor sampling rate, eliminates the influence of redundant and irrelevant features on model accuracy, and uses a single classification algorithm to train the user behavior model, which can improve the balance between security and usability of the authentication method.

[0073] Optionally, the pre-processed acceleration, the single vibration signal captured by the gyroscope sensor, and the touch screen touch data are used as inputs to analyze and extract a feature set of the click behavior pattern, including:

[0074] Extract behavioral characteristics that can reflect behavioral data, including the energy, duration, and click vibration characteristics corresponding to the time domain changes of the vibration signal captured by the motion sensor, as well as the position, pressure, contact area, and touch duration of the touch screen data;

[0075] A preset n-point vibration signal sequence is S = {s0, s1, ..., s n-1}, S can be a complete click vibration sequence or a sub-stage pressing sequence and a recovery sequence. The types of click vibration features extracted from the vibration signal sequence include time domain distribution features, energy-related features, and time-related features. Among them, the time domain distribution features are used to reflect the time domain statistical distribution characteristics of the vibration signal; the energy-related features are directly related to the strength characteristics of the user's click behavior. The energy-related features include the signal short-time energy STE and the power spectrum density energy PSD. The short-time energy reflects the time domain energy characteristics, and the calculation expression is: PSD is the distribution of power with frequency, which is used to reflect the energy characteristics in the frequency domain. The corresponding expression is: Where fs represents the sampling rate, and FFT() represents the fast Fourier transform; the time-related features are used to reflect the speed characteristics of the click behavior, the click behavior and the duration of different stages of the click;

[0076] According to each dimension of the motion sensor data and its two subsequences, ten vibration features are extracted from each subsequence, and six feature types are extracted from the complete sequence. Therefore, each set of motion sensor data can extract 2*10+6=26 dimensional features, and the X, Y, and Z three-dimensional data of the motion sensor can extract a total of 3*26=78 dimensions;

[0077] The energy amplitude sequence AMP of the motion sensor is not segmented, and ten features are directly extracted from it. 78+10=88-dimensional features can be extracted from one type of sensor data, and the total dimension of click vibration features extracted from the motion sensor modal data is 88*2=176.

[0078] In this embodiment, the extracted feature set is subjected to feature selection to characterize the key feature subset of the user behavior characteristics, including: using the Fisher score algorithm to measure the distinguishability of the feature based on the fact that the distance between the key features is small within the class and large between the classes, including the inter-class distance S B and intra-class distance S W The corresponding expressions of the two metrics are: Among them, S B (f k ), S w (f k ) represents each dimension feature f k Calculate the corresponding evaluation index, U represents the sample category set, n i represents the i-th class U i The number of samples, the fth k Wei Jiwei For U i Sample at f k The mean value on i is the proportion of samples of the i-th category in all samples, m k For all samples in the fth k The mean value on the dimension; assign a score J to each dimension feature Fisher (f k )=S B (f k ) / S w (f k ), S B With S w The ratio of is used to measure feature importance.

[0079] It should be noted that, since the system will also generate press and lift touch events when the touch screen detects touch screen sliding and multi-touch operation, other irrelevant data of the touch screen needs to be filtered. In the actual acquisition process, the touch screen modal data may be occasionally missing. These erroneous data can no longer be used to reflect user behavior habits, so the missing invalid data needs to be cleaned. Click vibration signal processing is the main goal of data preprocessing. According to the vibration signal analysis and touch click vibration processing process, including vibration extraction, signal interpolation and click process segmentation operations, the single click vibration signal sequence is divided into a press sequence and a recovery sequence. The two modal data of the touch screen and the motion sensor need to be associated and synchronized according to the timestamp attribute to achieve a one-to-one correspondence between the two modal data of a single touch click, so as to realize the subsequent fusion and splicing of the two modal features.

[0080] Optionally, the behavior authentication module includes:

[0081] Data preprocessing layer: When the user touches the screen, the sensor data is recorded and the raw data is obtained. The mean compensation algorithm is used to fill the missing values ​​of the collected raw data and filter out the abnormal values. Then, the discrete cosine transform is used to transform the raw data into the frequency domain to expand the feature space. Then, the noise components with too high and too low frequencies are filtered out to obtain a set of third-order tensors. The three dimensions of the third-order tensor are the sensor dimension, the sampling frequency of the original signal, and the frequency phase of the frequency domain signal.

[0082] Convolution layer: It includes an independent convolution layer corresponding to the third-order tensor of each sensor, a merged convolution layer that merges the independent convolution outputs of each sensor tensor, and a merged convolution layer that merges the independent convolution outputs of each sensor tensor. A multi-layer independent convolution layer structure is set to train and learn the internal relationship and extract the feature adjacent matrix. The matrix is ​​then converted into a one-dimensional adjacent matrix by splicing and passed to the merged convolution layer as input;

[0083] Recurrent layer: Input the one-dimensional feature vector output by the previous layer into a two-layer recurrent neural network and make an identity judgment for each user touch behavior;

[0084] Output layer: removes a number of judgment results obtained from the previous layer through a sliding window, and then inputs them into a fully connected neural network to give the final verification result.

[0085] As a preferred embodiment of the above technical solution, the original data becomes a set of d after data preprocessing. (k) *2f*N three-dimensional tensor, where d (k) represents the dimension of the kth sensor, f represents the number of frequency-phase pairs in the frequency domain, and N represents the number of input data matrices. represents the processed data matrix of the kth sensor in the nth touch cycle;

[0086] The data matrix As the input of an independent convolutional layer and perform feature extraction on multiple sensor data groups, the input matrix There is a relationship between frequency domain features and sensor dimension features that needs to be extracted, using a d (k) *c1's 2D convolution kernel learning matrix The correlation between the sensor measurement dimension and the local modes in the frequency domain is obtained, and the output matrix Enter the next convolutional layer;

[0087] According to the matrix Use two one-dimensional convolution kernels of 1*c2 and 1*c3 respectively to learn the abstract and high-level relationship between features, and get the output matrix and

[0088] The k-dimensional feature vector matrix generated by the features of multiple sensors is obtained through independent convolutional layers The eigenvector matrix The merged convolution layer is used as input, and the integrated feature vector matrix is ​​used for feature extraction;

[0089] Use a K*c4 two-dimensional convolution kernel to learn the intrinsic connection between K sensor feature vectors to get an output Use a one-dimensional 1*c5 convolution kernel and a one-dimensional 1*c6 convolution kernel to calculate and get and Among them, each convolution layer has a corresponding normalization operation to constrain the data to a fixed distribution range to avoid gradient vanishing and gradient explosion, and the expression of using ReLU as the activation function in each convolution layer is: After a concatenation operation, a one-dimensional feature vector x is obtained as the output of this convolutional layer.

[0090] In this embodiment, the mean square error is used as the loss function, and the Adam algorithm is used in the dropout layer to optimize the convergence speed. The corresponding expression is: Among them, m t represents the exponential decay bias of the first-order moment estimate, v t represents the exponential decay bias of the second-order moment estimate. β1 and β2 are the recommended default values ​​of the decay coefficients and are 0.9 and 0.999, respectively. t Represents the gradient; after initialization, the error is corrected and the corrected m is obtained by calculation t and v t To offset the deviation, the corresponding expression is Then the final gradient update rule is expressed as Where α represents the learning rate and ε represents the deviation compensation. The recommended default value is 10 -8 Each touch will generate a feature vector x after processing. It is assumed that there are N touches in the whole verification process to obtain a vector set X = {x1, x2...x N}, and input them into the proposed network structure in sequence to eventually generate the corresponding touch recognition results.

[0091] It should be noted that the sensor signal is preprocessed to filter out abnormal values ​​and interpolate missing values, and then the discrete cosine transform is used to convert it from time domain characteristics to frequency domain characteristics to expand the user behavior feature space. The convolution layer is used to extract the features collected on the smart device. Two layers of convolution are set up. Independent convolution extracts features from different sensor features. The merged convolution layer merges and integrates different sensor features, making the features better interpretable.

[0092] Optionally, the accelerometer triaxial data is processed and the amplitude M is used as the criterion for gait cycle segmentation. The expression of M is: The process of period segmentation based on the M value includes the following specific steps:

[0093] The local minimum value of M is calculated and used as the candidate set of the starting point of the gait cycle;

[0094] Calculate the average of all local minima and remove local minima above the average from the candidate set;

[0095] The candidate points that are adjacent in time are grouped together, and the minimum value in each group is selected as the starting point of the period segmentation. After the period segmentation is completed according to the M value, the period division time of the nine dimensions of the motion sensor measurement value is consistent with the starting point of M;

[0096] The accelerometer data is calibrated, and then the three-axis directions of the new coordinate system are obtained based on the calibrated data. η represents the X, Y, and Z axes of the user coordinate system respectively. The η axis of the coordinate system with the user as the reference is parallel to the user's torso. The axis is the front-to-back direction, and its positive direction is consistent with the user's movement direction. The γ axis is the left-to-right direction. The axis is perpendicular to the η axis,

[0097] Includes calibration of accelerometer data to user coordinate system γ, Generation of the η axis, the accelerometer data collected in the device coordinate system Ac′=[Ac′ X ,Ac′ Y ,Ac′ Z ] T Expressed as Where, Ac=[Acx ,Ac Y ,Ac Z ] T is the calibrated accelerometer vector, K is the sensitivity matrix; B = [b1, b2, b3] T is the deviation, n=[n1,n2,n3] T is a random noise close to 0;

[0098] By constructing the objective function as Where m represents the number of measurement locations, and g represents the local gravitational acceleration;

[0099] The gravity component contained in the sensor data collected by the device coordinate system is not fixed. The average gravity direction is used as the η axis of the user coordinate system. It is assumed that the current gait cycle is t (t = 1, 2...). In the gait cycle t, the gravity component g s It can be expressed as in, They represent the average values ​​of the three axes of the accelerometer in the gait cycle t respectively; the expression of the η axis in the user coordinate system is: ‖·‖ represents the modulus operation of the vector, and the accelerometer vector Ac = [Ac X ,Ac Y ,Ac Z ] T , gyroscope vector Gr = [Gr X ,Gr Y ,Gr Z ] T And the magnetometer vector Ma = [Ma X ,Ma T ,Ma Z ] T The projections on the η axis of the user coordinate system are Ac η , Gr η and Ma η , the corresponding calculation expression is

[0100] In this embodiment, the gravity component of the user coordinate system can be used to obtain its component on the horizontal plane. The positive direction of the axis is based on the PCA principle to find the direction with the largest variance in the original data. The direction is parallel to the user's movement direction, and the user coordinate system can be obtained. Axis, the specific algorithm steps include:

[0101] Removing the gravity component of the accelerometer, we get Ac h The average value of X, Y, and Z axis data Ac hThe three-axis data is decentralized, and the average value corresponding to each axis is subtracted from the value of each axis. Ac h The covariance matrix of Where n represents the number of samples in a gait cycle; the covariance matrix Cov(Ac h ), the eigenvalue and eigenvector of the maximum eigenvalue, and the eigenvector v corresponding to the maximum eigenvalue, v represents the eigenvalue in the user coordinate system. The direction corresponding to the axis is expressed as Accelerometer vector Ac = [Ac X ,Ac Y ,Ac Z ] T In the user coordinate system The projection onto the axis can be expressed as The corresponding gyroscope vector Gr = [Gr X ,Gr Y ,Gr Z ] T and magnetometer vector Ma = [Ma X ,Ma Y ,Ma Z ] T It can be expressed as

[0102] From the characteristics of the coordinate system, we know that the γ axis is The γ axis of the user coordinate system is obtained by calculating the cross product of the first two vectors: Then the accelerometer vector Ac=[Ac X ,Ac Y ,Ac Z ] T , gyroscope vector Gr = [Gr X ,Gr Y ,Gr Z ] T And the magnetometer vector Ma = [Ma X ,Ma Y ,Ma Z ] T The projection on the γ axis of the user coordinate system can be expressed as Then a sensor measurement value {Ac X ,Ac Y ,Ac Z ,Gr X ,Gr Y ,Gr Z ,Ma X ,Ma Y ,Ma Z} is eventually converted to

[0103] It should be noted that the motion sensor data embedded in the smartphone all refer to the device coordinate system of the phone itself. The X-axis of the device coordinate system is the left-right direction, the Y-axis is the up-down direction, and the Z-axis is perpendicular to the device screen direction. It can be seen that the data in this coordinate system is completely dependent on the direction of the phone itself. When the user's gait remains consistent and the phone rotates, the gait signal collected by the sensor also changes, and the data can be transformed into a more stable coordinate system. GRU is used to train the gait authentication model. GRU is a variant of RNN. It can overcome the problem that RNN is susceptible to gradient disappearance and gradient explosion during the training process. Compared with LSTM, it has fewer parameters and is easier to converge in training. The purpose of training the GRU gait authentication model is to solve the optimal model parameter θ (θ refers to all parameters in the model) for the current legitimate user. GRU uses the internal gating mechanism and storage unit of the unit to process the motion sensor sequence data. By continuously forward propagating and resetting the weights of the gate and updating the gate in reverse, the goal of minimizing the error between the true value and the predicted value is achieved.

[0104] See also Figure 2 The present invention also provides a smart phone security protection method based on the Internet, comprising the following steps:

[0105] S1: collecting behavioral data of the user's touch and click operations on the smartphone, and preprocessing the behavioral data, wherein the behavioral data includes motion sensor modal data and touch screen sensor modal data;

[0106] S2: using the pre-processed acceleration, the single vibration signal captured by the gyroscope sensor and the touch screen touch data as input to analyze and extract a feature set of the click behavior pattern, wherein the click behavior pattern includes force characteristics and speed characteristics;

[0107] S3: Perform feature selection on the extracted feature set to obtain a key feature subset that characterizes user behavior characteristics;

[0108] S4: training a behavior model corresponding to the behavior data using a single classifier for anomaly detection based on a subset of key features, and authenticating the legality of the behavior, wherein the behavior model includes legal behavior data and illegal behavior data;

[0109] S5: Acquire gait data of the user to be authenticated, and set an authentication mechanism based on the touch screen sensor modal data and the gait data to achieve security protection of the smartphone.

[0110] In this embodiment, during the authentication stage, after the gait data to be authenticated is preprocessed, the trained gait model outputs the probability that the current gait behavior belongs to a legitimate user, that is, the legitimate users and illegal users are classified into two categories. In the actual gait scenario, the gait behavior of a legitimate user may occasionally be judged as an illegal behavior, such as sudden fright, traffic conditions and other factors. If the smartphone is directly locked due to an abnormal gait behavior of a legitimate user, the false rejection rate of the authentication system increases, and it lacks flexibility. Therefore, in order to provide a better user experience, the authentication system has a certain fault tolerance. The authentication mechanism dynamically adjusts its trust value according to the gait authentication result of the current user. When the trust value is lower than the threshold, the phone is locked; otherwise, the user can use the phone normally. The specific process includes:

[0111] Step 1: After completing the registration of the user model, when the legitimate user unlocks the smartphone, the trust value V is initialized b =0; Step 2: The smartphone continuously monitors the user's gait behavior and collects motion sensor data; Step 3: The collected motion sensor data is subjected to preprocessing operations such as noise reduction, gait cycle segmentation, and coordinate system conversion; Step 4: For the gait data x of user j j , and obtain the probability P(y j |x j ,θ), and use this probability as the gait authentication score s g , where θ represents the parameters of model training, y j represents the true label of user j, and the score s g Compared with the predetermined discrimination threshold τ, if s g ≥τ, then y j =1, authentication passed; otherwise, if s g <τ, then y j =0, authentication failed; Step 5: If the authentication is successful, it means that the current gait behavior is judged to be a legal behavior, and the trust value V b Increase, the user can use the phone normally and return to step 2 to continue execution; if the authentication fails, it means that the current gait behavior is illegal, and the trust value V b Lower, V b The mechanism of increase and decrease is expressed as Among them, V′ bDenote the previous trust value as \(V_{prev}\), and \(k\) represents the number of consecutive authentication failures of the illegal user. If the current behavior is legal, the trust value of the user increases, and the value of \(k\) is reset to 0; otherwise, if it is an illegal behavior, the trust value of the current user decreases, and the value of \(k\) is incremented by 1. It is worth noting that the more times of illegal behavior, the more exponentially the trust value of the current user decreases. The purpose is that when a legitimate user is using a smart phone and has obtained a relatively high trust value, if an illegal user accesses it, by reducing the trust value at an exponential rate, the phone can be quickly locked to prevent the illegal user from accessing private data; Step 6: Compare the trust value \(V\) b with the trust threshold \(V_0\). Set the trust threshold to 0. If \(V\) b ≥ \(V_0\), the phone can be used normally, and return to Step 2 for continuous monitoring by the system; if \(V\) b < \(V_0\), the phone is locked, and the current user needs to re-authenticate through explicit authentication methods such as passwords. Set the upper limit of the trust value \(V\) b to 10 to prevent the situation that when a legitimate user's multiple legal behaviors increase the trust value to a large value, an illegal user invades, and at this time the system cannot reduce the trust value below the threshold in a short period, resulting in more time for the illegal user to perform illegal behaviors.

[0112] In all the examples shown and described here, any specific value should be interpreted as merely exemplary and not as a limitation. Therefore, other examples of the exemplary embodiments may have different values.

[0113] It should be noted that: Similar reference numerals and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0114] The above-described embodiments merely represent several implementation manners of the present invention. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the present invention. It should be pointed out that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all belong to the protection scope of the present invention.

Claims

1. A smart phone security protection system based on the Internet, characterized in that: include: A data collection module is used to collect the behavior data of the user's touch and click operations on the smart phone and pre-process the behavior data, wherein the behavior data includes motion sensor modal data and touch screen sensor modal data; A feature extraction module is used to analyze and extract a feature set of a click behavior pattern using the preprocessed acceleration, a single vibration signal captured by a gyroscope sensor, and touch data of the touch screen as input, wherein the click behavior pattern includes a force characteristic and a speed characteristic; A feature selection module is used to select the extracted feature set to obtain a key feature subset that characterizes the user behavior characteristics; A behavior authentication module is used to train a behavior model corresponding to the behavior data using a single classifier for anomaly detection based on a subset of key features, and authenticate the legality of the behavior, wherein the behavior model includes legal behavior data and illegal behavior data; Identity decision evaluation is used to obtain the gait data of the user to be authenticated, and to set up an authentication mechanism based on the touch screen sensor modal data and gait data to achieve security protection for smartphones.

2. The Internet-based smartphone security protection system according to claim 1, characterized in that: Collect the behavior data of touch and click operations on smartphones and pre-process the behavior data, including: Each sampling point MS of the motion sensor modal data i Including X, Y, Z three-dimensional data of accelerometer acc and gyroscope gyro, sampling timestamp is TmSp, touch occurrence flag TaFg, the corresponding sampling point expression is: MS i =[accX i ,accY i ,accZ i ,gyoX i ,gyoY i ,gyroZ i ,TmSp i ,TaF i ], the touch screen modal data of a single touch click includes two touch events at the time of pressing and lifting, recorded as TM = [TE down ,TE up ], the single touch event data structure TE is expressed as TE = [posX, posY, press, major, minor, oritn, EvTe], with the upper left corner of the touch screen as the origin, posX and posY as the horizontal and vertical coordinates of the touch point, press as the pressure applied by the finger, major and minor as the major and minor axes of the elliptical contact surface between the finger and the touch screen, oritn as the direction of the contact surface, EvTe indicates the occurrence of a touch event, and the motion sensor modal data and the touch screen sensor modal data are associated and synchronized according to their respective recorded timestamps to achieve a one-to-one correspondence between the motion sensor modality and the touch screen modality of a single click; When a touch click operation occurs, the touch occurrence flag TaFg of the motion sensor sampling point generates a continuous "1" to indicate that this signal sequence is a click vibration sequence, and the click vibration signal is preprocessed, extracted and segmented to generate a user behavior data set, where data preprocessing includes abnormal data cleaning, click vibration processing and modal data association.

3. The Internet-based smart phone security protection system according to claim 1, characterized in that: The pre-processed acceleration, single vibration signal captured by the gyroscope sensor and touch screen touch data are used as input to analyze and extract the feature set of click behavior patterns, including: Extract behavioral characteristics that can reflect behavioral data, including the energy, duration, and click vibration characteristics corresponding to the time domain changes of the vibration signal captured by the motion sensor, as well as the position, pressure, contact area, and touch duration of the touch screen data; A preset n-point vibration signal sequence is S = {s0, s1, ..., s n-1 }, S can be a complete click vibration sequence or a sub-stage pressing sequence and a recovery sequence. The types of click vibration features extracted from the vibration signal sequence include time domain distribution features, energy-related features, and time-related features. Among them, the time domain distribution features are used to reflect the time domain statistical distribution characteristics of the vibration signal; the energy-related features are directly related to the strength characteristics of the user's click behavior. The energy-related features include the signal short-time energy STE and the power spectrum density energy PSD. The short-time energy reflects the time domain energy characteristics, and the calculation expression is: PSD is the distribution of power with frequency, which is used to reflect the energy characteristics in the frequency domain. The corresponding expression is: Where fs represents the sampling rate, and FFT() represents the fast Fourier transform; the time-related features are used to reflect the speed characteristics of the click behavior, the click behavior and the duration of different stages of the click; According to each dimension of the motion sensor data and its two subsequences, ten vibration features are extracted from each subsequence, and six feature types are extracted from the complete sequence. Therefore, each set of motion sensor data can extract 2*10+6=26 dimensional features, and the X, Y, and Z three-dimensional data of the motion sensor can extract a total of 3*26=78 dimensions; The energy amplitude sequence AMP of the motion sensor is not segmented, and ten features are directly extracted from it. 78+10=88-dimensional features can be extracted from one type of sensor data, and the total dimension of click vibration features extracted from the motion sensor modal data is 88*2=176.

4. The Internet-based smart phone security protection system according to claim 1, characterized in that: The extracted feature set is subjected to feature selection to characterize the key feature subsets of user behavior characteristics, including: The Fisher score algorithm is used to measure the distinguishability of features based on the fact that the distance between key features is small within the class and large between classes, including the distance between classes S B and intra-class distance S W The corresponding expressions of the two metrics are: Among them, S B (f k ), S w (f k ) represents each dimension feature f k Calculate the corresponding evaluation index, U represents the sample category set, n i represents the i-th class U i The number of samples, the fth k Wei Jiwei For U i Sample at f k The mean value on i is the proportion of samples of the i-th category in all samples, m k For all samples in the fth k The mean value over the dimension; Assign a score J to each dimension feature Fisher (f k )=S B (f k ) / S w (f k ), S B With S w The ratio of is used to measure feature importance.

5. The Internet-based smart phone security protection system according to claim 1, characterized in that: The behavioral authentication module includes: Data preprocessing layer: When the user touches the screen, the sensor data is recorded and the raw data is obtained. The mean compensation algorithm is used to fill the missing values ​​of the collected raw data and filter out the abnormal values. Then, the discrete cosine transform is used to transform the raw data into the frequency domain to expand the feature space. Then, the noise components with too high and too low frequencies are filtered out to obtain a set of third-order tensors. The three dimensions of the third-order tensor are the sensor dimension, the sampling frequency of the original signal, and the frequency phase of the frequency domain signal. Convolution layer: It includes an independent convolution layer corresponding to the third-order tensor of each sensor, a merged convolution layer that merges the independent convolution outputs of each sensor tensor, and a merged convolution layer that merges the independent convolution outputs of each sensor tensor. A multi-layer independent convolution layer structure is set to train and learn the internal relationship and extract the feature adjacent matrix. The matrix is ​​then converted into a one-dimensional adjacent matrix by splicing and passed to the merged convolution layer as input; Recurrent layer: Input the one-dimensional feature vector output by the previous layer into a two-layer recurrent neural network and make an identity judgment for each user touch behavior; Output layer: removes a number of judgment results obtained from the previous layer through a sliding window, and then inputs them into a fully connected neural network to give the final verification result.

6. The Internet-based smart phone security protection system according to claim 5, characterized in that: After data preprocessing, the original data becomes a set of d (k) *2f*N three-dimensional tensor, where d (k) represents the dimension of the kth sensor, f represents the number of frequency-phase pairs in the frequency domain, and N represents the number of input data matrices. represents the processed data matrix of the kth sensor in the nth touch cycle; The data matrix As the input of an independent convolutional layer and perform feature extraction on multiple sensor data groups, the input matrix There is a relationship between frequency domain features and sensor dimension features that needs to be extracted, using a d (k) *c1's 2D convolution kernel learning matrix The correlation between the sensor measurement dimension and the local modes in the frequency domain is obtained, and the output matrix Enter the next convolutional layer; According to the matrix Use two one-dimensional convolution kernels of 1*c2 and 1*c3 respectively to learn the abstract and high-level relationship between features, and get the output matrix and The k-dimensional feature vector matrix generated by the features of multiple sensors is obtained through independent convolutional layers The eigenvector matrix The merged convolution layer is used as input, and the integrated feature vector matrix is ​​used for feature extraction; Use a K*c4 two-dimensional convolution kernel to learn the intrinsic connection between K sensor feature vectors to get an output Use a one-dimensional 1*c5 convolution kernel and a one-dimensional 1*c6 convolution kernel to calculate and get and Among them, each convolution layer has a corresponding normalization operation to constrain the data to a fixed distribution range to avoid gradient vanishing and gradient explosion, and the expression of using ReLU as the activation function in each convolution layer is: After a concatenation operation, a one-dimensional feature vector x is obtained as the output of this convolutional layer.

7. The Internet-based smart phone security protection system according to claim 5, characterized in that: The mean square error is used as the loss function, and the Adam algorithm is used in the Droupout layer to optimize the convergence speed. The corresponding expression is Among them, m t represents the exponentially decaying bias of the first-order moment estimate, v t represents the exponential decay bias of the second-order moment estimate. β1 and β2 are the recommended default values ​​of the decay coefficients and are 0.9 and 0.999, respectively. t represents the gradient; After initialization, the error is corrected and the corrected m is obtained by calculation. t and v t To offset the deviation, the corresponding expression is Then the final gradient update rule is expressed as Where α represents the learning rate and v represents the deviation compensation. The recommended default value is 10 -8 ; Each touch will generate a feature vector x after processing. It is assumed that there are N touches in the whole verification process to obtain a vector set X = {x1, x2...x N }, and input them into the proposed network structure in sequence to eventually generate the corresponding touch recognition results.

8. The Internet-based smart phone security protection system according to claim 1, characterized in that: The accelerometer triaxial data is processed and the amplitude M is used as the criterion for gait cycle segmentation. The expression of M is: The process of period segmentation based on the M value includes the following specific steps: The local minimum value of M is calculated and used as the candidate set of the starting point of the gait cycle; Calculate the average of all local minima and remove local minima above the average from the candidate set; The candidate points that are adjacent in time are grouped together, and the minimum value in each group is selected as the starting point of the period segmentation. After the period segmentation is completed according to the M value, the period division time of the nine dimensions of the motion sensor measurement value is consistent with the starting point of M; The accelerometer data is calibrated, and then the three-axis directions of the new coordinate system are obtained based on the calibrated data. η represents the X, Y, and Z axes of the user coordinate system respectively. The η axis of the coordinate system with the user as the reference is parallel to the user's torso. The axis is the front-to-back direction, and its positive direction is consistent with the user's movement direction. The γ axis is the left-to-right direction. The axis is perpendicular to the η axis, Includes calibration of accelerometer data to user coordinate system γ, Generation of the η axis, the accelerometer data collected in the device coordinate system Ac′=[Ac′ X ,Ac′ Y ,Ac′ Z ] T Expressed as Where, Ac=[Ac x ,Ac Y ,Ac Z ] T is the calibrated accelerometer vector, K is the sensitivity matrix; B = [b1, b2, b3] T is the deviation, n=[n1,n2,n3] T is a random noise close to 0; By constructing the objective function as Where m represents the number of measurement locations, and g represents the local gravitational acceleration; The gravity component contained in the sensor data collected by the device coordinate system is not fixed. The average gravity direction is used as the η axis of the user coordinate system. It is assumed that the current gait cycle is t (t = 1, 2...). In the gait cycle t, the gravity component g s It can be expressed as in, They represent the average values ​​of the three axes of the accelerometer in the gait cycle t respectively; the expression of the η axis in the user coordinate system is: ‖·‖ represents the modulus operation of the vector, and the accelerometer vector Ac = [Ac X ,Ac Y ,Ac Z ] T , gyroscope vector Gr = [Gr X ,Gr Y ,Gr Z ] T And the magnetometer vector Ma = [Ma X ,Ma T ,Ma Z ] T The projections on the η axis of the user coordinate system are Ac η , Gr η and Ma η , the corresponding calculation expression is 9. The Internet-based smart phone security protection system according to claim 8, characterized in that: The gravity component of the user coordinate system can be used to obtain its component on the horizontal plane. The positive direction of the axis is based on the PCA principle to find the direction with the largest variance in the original data. The direction is parallel to the user's movement direction, and the user coordinate system can be obtained. Axis, the specific algorithm steps include: Removing the gravity component of the accelerometer, we get Ac h The average value of X, Y, and Z axis data Ac h The three-axis data is decentralized, and the average value corresponding to each axis is subtracted from the value of each axis. h Ac h The covariance matrix of Where n represents the number of samples in a gait cycle; Get the covariance matrix Cov(Ac h ), the eigenvalue and eigenvector of the maximum eigenvalue, and the eigenvector v corresponding to the maximum eigenvalue, v represents the eigenvalue in the user coordinate system. The direction corresponding to the axis is expressed as Accelerometer vector Ac = [Ac X ,Ac Y ,Ac Z ] T In the user coordinate system The projection onto the axis can be expressed as The corresponding gyroscope vector Gr = [Gr X ,Gr Y ,Gr Z ] T and magnetometer vector Ma = [Ma X ,Ma Y ,Ma Z ] T It can be expressed as From the characteristics of the coordinate system, we know that the γ axis is The γ axis of the user coordinate system is obtained by calculating the cross product of the first two vectors: Then the accelerometer vector Ac=[Ac X ,Ac Y ,Ac Z ] T , gyroscope vector Gr = [Gr X ,Gr Y ,Gr Z ] T And the magnetometer vector Ma = [Ma X ,Ma Y ,Ma Z ] T The projection on the γ axis of the user coordinate system can be expressed as Then a sensor measurement value {Ac X ,Ac Y ,Ac Z ,Gr X ,Gr Y ,Gr Z ,Ma X ,Ma Y ,Ma Z } is eventually converted to 10. A method for protecting a smart phone based on the Internet according to the smart phone security protection system based on the Internet according to any one of claims 1 to 9, characterized in that: The following steps are involved: Collecting behavioral data of the user's touch and click operations on the smartphone and preprocessing the behavioral data, wherein the behavioral data includes motion sensor modal data and touch screen sensor modal data; The pre-processed acceleration, the single vibration signal captured by the gyroscope sensor and the touch screen touch data are used as input to analyze and extract the feature set of the click behavior pattern, wherein the click behavior pattern includes force characteristics and speed characteristics; The extracted feature set is subjected to feature selection to obtain a key feature subset that characterizes user behavior characteristics; A single classifier for anomaly detection is used to train a behavior model corresponding to the behavior data according to a subset of key features, and the legality of the behavior is authenticated, wherein the behavior model includes legal behavior data and illegal behavior data; The gait data of the user to be authenticated is obtained, and an authentication mechanism is set based on the touch screen sensor modal data and gait data to achieve security protection of the smartphone.

Images