A multi-scene subject recognition method based on keystroke dynamics and mouse dynamics

Abstract

A multi-scenario subject recognition method based on keystroke dynamics and mouse dynamics belongs to the field of biometrics. The present invention models a model that can identify users in free scenes through several special scenes of daily computers. In the modeling stage, users can collect data based on several specific scenes and then model their own scenes. This overcomes the limitations of single-scene modeling and the long data collection process required for free scene modeling.

Description

Technical Field

[0001] The invention belongs to the field of biometrics, and in particular relates to a subject recognition method based on keystroke dynamics and mouse dynamics. Background Art

[0002] Biometrics is a technology that combines computers with high-tech technologies such as optics, acoustics, and biostatistics, and uses the inherent physiological characteristics of the human body (such as fingerprints, irises, etc.) and behavioral characteristics (such as voice, keystroke habits, mouse usage habits, etc.) to perform identity authentication. Compared with traditional subject identification methods (such as keys, user names and passwords, etc.), biometrics has the advantages of being difficult to forget, good anti-counterfeiting, and difficult to forge. In addition, the successful commercial use of biometrics based on physiological and behavioral characteristics such as fingerprints, irises, and voices proves that this field has a longer-term development prospect. The biometric system is an access control system in which users are divided into legal users and illegal users. Legal users are people who are authenticated to use the system, and illegal users are people who cannot use the system. The role of the biometric system is to allow all legal users to access the system normally, and illegal users cannot access the system (even if illegal users want to disguise themselves as legal users).

[0003] Keystroke dynamics is the analysis of a person's typing habits. Keystroke dynamics is not about what the user types, but how they type, such as how long a key is pressed or how long it takes between keystrokes. Looking closely at these habits can produce patterns or signatures that are unique to each person. Keystroke dynamics is a form of biometrics that, unlike passwords, verifies a person's identity by how they type. Typing habits are not easily intercepted or stolen, so keystroke dynamics is an excellent user authentication scheme that can be added to a regular ID / password verification scheme.

[0004] Mouse dynamics was proposed later than keystroke dynamics. It is an analysis of a person's mouse usage habits. It studies features such as the average mouse movement speed in all directions, the time interval between single and double clicks, etc.

[0005] Identification by using physiological features such as fingerprints, faces, and facial features has been widely used in daily electronic products. Although it has advantages such as not easy to forget compared with traditional identity authentication methods. But in the final analysis, physiological feature recognition, like traditional identity authentication, is a single-time authentication technology. If the system is not shut down after the system is verified, anyone (including illegal users) can use the system, that is, physiological feature recognition does not have the ability of continuous authentication. On the contrary, the identity authentication system based on keystroke dynamics and mouse dynamics can continue to authenticate the user after the user logs in to the system. If the system identifies an illegal user, it will actively shut down and require the user to authenticate again. In real scenarios, for example, if the user logs in and forgets to lock the system after leaving, if only a single authentication is used, the user's data security cannot be guaranteed. Therefore, using keystroke dynamics and mouse dynamics for continuous user identity authentication is an important part of ensuring data security. In addition, thanks to the fact that both technologies are based on existing external devices of the computer, the authentication method based on keystroke dynamics and mouse dynamics is cheaper than other authentication methods, and thus has stronger universality.

[0006] There are currently two types of user authentication methods for keystroke dynamics, one is a fixed text authentication method combined with a password, and the other is a random text authentication method during user use. The fixed text authentication method ensures that illegal users cannot enter the system by typing in passwords if the user password is stolen, and the random text authentication method implements continuous authentication of users after logging into the system. Both methods have their own advantages and disadvantages. Combining the two authentication methods can more comprehensively protect data security. There are also two types of user authentication methods for mouse dynamics, namely static detection method and continuous detection method. The static detection method is to detect users within a specific time (for example, when users log in with passwords), and the continuous detection method implements continuous authentication of users after logging into the system. Like keystroke dynamics, using static detection and continuous detection methods at the same time can better protect data security.

[0007] Most keystroke and mouse dynamics authentication methods are limited to one of the two, and are limited to a single scenario (such as typing) and real-world scenarios (normal computer use in daily life). After 2011, methods that combine keystroke and mouse dynamics became popular. Using keystroke and mouse features at the same time can complement each other while ensuring their respective performance, thereby achieving better performance. Mouse behavior variability can be reduced through feature selection. Some studies have applied multidimensional scaling and Laplacian eigenmaps to mouse feature selection, successfully reducing mouse behavior variability.

[0008] Although there are already some promising methods for subject recognition systems based on keystroke dynamics, mouse dynamics, and the fusion of the two, current research focuses on single scenes and free scenes. However, both of these scenes have certain limitations in practical applications. Single scenes have weak universality, and free scenes require more data to be collected, resulting in a longer modeling time. Summary of the invention

[0009] The purpose of the present invention is to build a model that can identify users in free scenes through several special scenes of daily computers. During the modeling stage, users can collect data based on several specific scenes and then conduct their own scene modeling. This overcomes the limitations of single-scene modeling and the long data collection process required for free scene modeling.

[0010] 1. A multi-scene subject recognition method based on keystroke dynamics and mouse dynamics, characterized by comprising:

[0011] Step 1: Feature extraction and data preprocessing;

[0012] Step 2: Generate feature weight vectors for each scene using mutual information and information entropy methods, and merge the weight vectors into a weight matrix;

[0013] Step 3: Use support vector machine and weight matrix regularization to unify the weights of each scene;

[0014] Step 4: Select features based on the weight matrix and use long short-term memory artificial neural network for user identification.

[0015] Step 1: Extract features and preprocess data

[0016] The timing features of the keyboard and mouse are extracted according to the length of the time slice; there are 5 types of keyboard timing features, namely the time interval between pressing and releasing a single key (DT), the time interval between releasing the first key and pressing the second key (UD), the time interval between pressing the first key and pressing the second key (DD), the time interval between releasing the first key and releasing the second key (UU), and the time interval between pressing the first key and releasing the second key (DU); the keystroke feature is applicable to each key and each key pair. For a full-size 110-key keyboard, the DT feature of a single key in each time slice has 110 dimensions, and the UD, DD, UU, and DU features between key pairs have 12100 dimensions each, totaling 48400 dimensions. Therefore, the keystroke feature in each time slice has 48510 dimensions; there are 7 types of mouse timing features, namely the average movement speed of each distance divided by 100 pixels (MDS), the average movement speed of each direction (MDA), the proportion of the number of operations in each direction (MDH), the average movement speed of each operation type (ATA), and the operation of each operation type. The operation number ratio (ATH), the operation number ratio of each moving distance (TDH) and the operation number ratio (MTH) in each time; the average moving speed of each distance (MDS), the average moving speed of each direction (MDA), the operation number ratio of each direction (MDH) and the operation number ratio (TDH) of each moving distance are defined according to 8 directions and every 100 pixel spacing within 1 to 800 pixels, so they are 8-dimensional features; the average moving speed of each operation type (ATA) and the operation number ratio (ATH) of each operation type are defined according to the four operation types of the user (moving, dragging, prohibiting and clicking), so they are 4-dimensional features; the operation number ratio (MTH) in each time is the time interval of every 200 milliseconds within 1 to 1000 milliseconds, so it is a 5-dimensional feature; finally, the mouse feature in each time slice is 45 dimensions in total, and the feature vector obtained in each time slice has 45 elements; after obtaining the keystroke feature and the mouse feature respectively, they are spliced ​​according to the time slice to obtain a 48555-dimensional feature vector, and the corresponding feature components that do not appear in a single time slice are filled with 0;

[0017] The eigenvectors with all 0 eigencomponents in a single time slice and the eigencomponents with all 0 in all eigenvectors are deleted, and the data preprocessing is completed using the homogeneous mean interpolation method;

[0018] Step 2: Use information entropy and mutual information methods to generate feature weight vectors for each scene, and merge the weight vectors into a weight matrix

[0019] The eigenvector is defined as f = [f1,f2,f3,f4,...,f n-1 ,f n ], where each f i is a characteristic component, and n can be any number;

[0020] Information entropy is a quantitative measure of information, that is, the expected amount of information about an event; its definition formula is

[0021]

[0022] Where p(x i ) is a random event x i Probability of occurrence;

[0023] Based on the above information entropy definition formula, each feature component f is calculated i Information entropy

[0024]

[0025] in |D f | is the number of instances in the data set (i.e., the feature component f i The number of ij is the value of the characteristic component, |a ij | is the characteristic component f in each eigenvector i The value is a ij The number of

[0026] The mutual information definition formula is:

[0027]

[0028] Where p(x, y) is the joint probability distribution of random variables X and Y, p(x) and p(y) are the marginal probability distributions of random variables X and Y respectively;

[0029] Calculate each eigenvalue f i With other characteristic components f j Mutual Information of (i≠j)

[0030]

[0031] The joint probability distribution |D ij | is the characteristic component f i The number of eigenvalues ​​f j The number of ik , a jl )| is the characteristic component f i The median value is a ik The number and characteristic components f j The median value is a jl The product of the number of ; marginal probability distribution What is obtained here is the mutual information between each feature component. In order to obtain the mutual information of a single feature component relative to all feature components, the average normalized mutual information is used here

[0032]

[0033] Where I(f i ;f j ) is the characteristic component f i and f j The mutual information between i ) and H(f j ) are the characteristic components f i and f j The information entropy of ; the average normalized mutual information reflects the characteristic component f i The overall mutual information size with all other feature components;

[0034] According to the amount of information within and between feature components reflected by information entropy and average normalized mutual information, the weight of feature components can be obtained by subtracting information entropy from average normalized mutual information, that is, using the formula

[0035] W(f i )=H(f i )-AVGMI NORM (f i ), (6)

[0036] Where H(f i ) is the characteristic component f i Information entropy, AVGMI NORM (f i ) is the characteristic component f i The average normalized mutual information of

[0037] After obtaining the feature component weights of each scene according to formula (6), each scene can generate a 48555-dimensional feature weight column vector. The feature weight column vectors are horizontally spliced ​​and all zero rows are deleted to obtain the weight matrix W 权重向量维数×4 , all 0 rows mean that the feature component has no contribution to classification in each scene; at this time, each column of the feature weight matrix is ​​the importance information of each scene for each feature component, and each row is the importance of the same feature component in each scene;

[0038] Step 3: Use support vector machine and weight matrix regularization to unify the weights of each scene

[0039] The most commonly used function in support vector machines is the hinge loss function, which is expressed as

[0040]

[0041] in is the predicted value of the sample, y∈{-1,1} is the true value of the sample, and the max() function takes 0 and The maximum value of, that is, the loss function is 0 when the classification is correct, otherwise it is

[0042] Use support vector machines as classifiers in the weight matrix regularization process;

[0043] Apply the weight matrix l to the objective function 2,1 Norm regularization; l 2,1 The definition formula of norm is

[0044]

[0045] where X is an r-row and t-column matrix, x ij is the i-th row and j-th column element of matrix X;

[0046] See l through the formula 2,1 The norm is the l1 norm of the l2 norm of each row of the matrix; l 2,1 The norm ensures the row sparsity of the matrix while ensuring the density of each row and column of the matrix. For the weight matrix used, ensuring the column density of each row ensures the convergence of the weights of the same feature component in the four scenes, and ensuring the column sparsity ensures that the weights of the feature components that are important to the four tasks are increased, and the weights of the unimportant ones are reduced; therefore, the objective function is used

[0047]

[0048] Get the target weight matrix, where u is the number of scenes, w is the number of users, loss(·) is the loss function, α is the regularization coefficient, and w i is the feature weight of the i-th scene, is the feature vector of the jth user in the i-th scene, is the label of the jth user in the i-th scene, ||W|| 2，1 is the feature weight moment W 2，1 norm;

[0049] Use the hinge loss function and determine the learning rate and regularization coefficient through the following steps: first set the regularization coefficient to 0, and take 10 numbers at equal intervals as the learning rate in four value ranges [0.0001, 0.001), [0.001, 0.01), [0.01, 0.1), [0.1, 1), and use these learning rates for learning, and iterate respectively. After that, the loss values ​​are obtained and these loss values ​​are applied to draw the loss value curve. The learning rate corresponding to the lowest point of the curve is selected as the learning rate during application. After determining the learning rate, the regularization coefficient is adjusted according to the accuracy of the test set. First, the order of magnitude of the regularization coefficient is determined, and the regularization coefficient is set to 1. The regularization coefficient is changed according to the accuracy of the test set. If the accuracy of the test set is lower than the accuracy of the training set, the regularization coefficient is increased by 10 times. If the accuracy of the test set is higher than the accuracy of the training set, the regularization coefficient is reduced by 10 times. A regularization coefficient order of magnitude with good test set accuracy and training set accuracy is obtained, and the regularization coefficient is increased under this order of magnitude. The coefficient with the best test set accuracy is selected as the final regularization coefficient α.

[0050] After determining the learning rate and regularization coefficient, the objective function is iterated for four scenarios at most. times, and the model accuracy no longer improves so that the objective function converges to obtain l 2， The weight matrix after norm regularization;

[0051] Step 4: Select features based on the weight matrix and use the long short-term memory artificial neural network for user identification

[0052] Take the average value of each row in the feature weight matrix and transpose it to obtain the multi-task average feature weight row vector AVGW 1 ×权重向量维数(d) =[w 0 , ..., w d ], the feature selection method uses the formula

[0053]

[0054] where w i , w j is the component of the average feature weight vector AVGW, threshold is the threshold, is the sum of all feature component weights, It is the sum of the weights of the m largest feature components, m starts from 0 and increases by 1 each time until the ratio is greater than the threshold, that is, the m largest feature components are selected as the input of the long short-term memory artificial neural network; here the threshold is set to 0.95 to obtain the m largest feature components;

[0055] Long short-term memory artificial neural network LSTM

[0056] In LSTM, the current input and the previous output are concatenated and multiplied by the input gate, output gate, and forget gate weight vector W respectively. i , W o , W f, and the corresponding control signals are obtained after the activation function Sigmoid, and the spliced ​​information is activated by the tanh function to obtain the input data to be processed At this moment, the forget gate control signal f t and the hidden layer information C at the previous moment t-1 The multiplication result and the input gate control signal i t With input data to be processed The multiplication results are added to obtain the hidden layer information at the current moment. The hidden layer information at the current moment is activated by the tanh function (output [-1, 1]) and combined with the output gate control signal O t Multiply to get the current output signal h t .

[0057] The data of four scenes are used in the examples of the present invention, namely document editing, shopping on Taobao, shopping on Weibo, and playing games. A self-written program is used to collect data under the conditions of a fixed computer and fixed peripherals, and the keystroke and mouse features of the four scenes are extracted respectively through a feature extraction program. The features extracted from the four scenes are spliced, and the weights are calculated by subtracting mutual information and information entropy. After obtaining the weights of the four scenes, they are merged into a weight matrix and the weights are multiplied by the eigenvalues ​​and respectively sent to the corresponding recursive neural network (RNN). The weight matrix is ​​regularized, and according to the sparsity between the rows and columns of the matrix, a feature subset common to the scenes for free scene recognition can be obtained.

[0058] Using corresponding feature subsets during training and recognition can achieve the expansion of multiple single scene recognition to free scene recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 Flowchart of the present invention

[0060] Figure 2 Keystroke dynamics

[0061] Figure 3 Mouse dynamics

[0062] Figure 4 LSTM Internal Structure DETAILED DESCRIPTION

[0063] like Figure 1As shown, the present invention provides a multi-scenario subject recognition method based on keystroke dynamics and mouse dynamics, including: step 1, feature extraction, data preprocessing; step 2, using mutual information and information entropy methods to generate feature weight vectors for each scene, and merging the weight vectors into a weight matrix; step 3, using support vector machines and weight matrix regularization to unify the weights of each scene; step 4, selecting features according to the weight matrix, and using long short-term memory artificial neural network for user recognition.

[0064] Step 1: Extract features and preprocess data

[0065] Feature extraction

[0066] Feature extraction refers to the method and process of extracting characteristic information from measurement data by computer. Its purpose is to generate a set of data that can reflect the characteristics of the data to promote subsequent learning and generalization. The more commonly used data features include time series features, rhythmic features, etc.

[0067] The present invention extracts the timing features of the keyboard and mouse according to the length of the time slice. There are five types of keyboard timing features, namely, the time interval between the pressing and releasing of a single key (DT), the time interval between the release of the first key and the pressing of the second key (UD), the time interval between the pressing of the first key and the pressing of the second key (DD), the time interval between the release of the first key and the release of the second key (UU), and the time interval between the pressing of the first key and the release of the second key (DU). The keystroke feature is applicable to each key and each key pair. For a full-size 110-key keyboard, the DT feature of a single key in each time slice has a total of 110 dimensions, and the UD, DD, UU, and DU features between key pairs have 12100 dimensions each, for a total of 48400 dimensions, so the keystroke feature in each time slice has a total of 48510 dimensions. There are 7 types of mouse timing features, namely, the average moving speed (MDS) of each distance divided by 100 pixels, the average moving speed (MDA) of each direction, the ratio of the number of operations in each direction (MDH), the average moving speed (ATA) of each operation type, the ratio of the number of operations of each operation type (ATH), the ratio of the number of operations of each moving distance (TDH), and the ratio of the number of operations in each time (MTH). Among them, the average moving speed (MDS) of each distance, the average moving speed (MDA) of each direction, the ratio of the number of operations in each direction (MDH), and the ratio of the number of operations of each moving distance (TDH) are defined according to the 8 directions and the spacing of every 100 pixels within 1 to 800 pixels, so they are 8-dimensional features. The average moving speed (ATA) of each operation type and the ratio of the number of operations of each operation type (ATH) are defined according to the four types of operations of the user (moving, dragging, prohibiting, and clicking), so they are 4-dimensional features. The ratio of the number of operations in each time (MTH) is the time interval of every 200 milliseconds within 1 to 1000 milliseconds, so it is a 5-dimensional feature. Finally, the mouse features in each time slice are 45-dimensional in total (the feature vector obtained in each time slice has 45 elements). After obtaining the keystroke features and mouse features respectively, they are spliced ​​according to the time slice to obtain a 48555-dimensional feature vector (the corresponding feature components that do not appear in a single time slice are filled with 0).

[0068] Data preprocessing

[0069] When using real data obtained daily for engineering practice, the data often contains some missing values, outliers, etc. At this time, it is necessary to apply data preprocessing (Data Preprocessing) to process the data before inputting it into the machine learning model. The quality of the preprocessing method has a great impact on the performance of the machine learning model.

[0070] The present invention deletes the eigenvectors whose eigencomponents are all 0 in a single time slice and the eigencomponents whose eigencomponents are all 0 in all eigenvectors, and uses the same type mean interpolation method (replacing according to the average value of the 0-valued eigencomponents in the same type according to the classification of the samples) to complete data preprocessing.

[0071] Step 2: Use information entropy and mutual information methods to generate feature weight vectors for each scene, and merge the weight vectors into a weight matrix

[0072] In the present invention, the feature vector is defined as f=[f1, f2, f3, f4, ..., f n-1 , f n ], where each f i is a characteristic component, and n can be any number.

[0073] Information entropy

[0074] Information entropy is a quantitative measure of information, that is, the expected amount of information about an event. Its definition formula is:

[0075]

[0076] Where p(x i ) is a random event x i Probability of occurrence.

[0077] In the present invention, each characteristic component f is calculated based on the above information entropy definition formula i Information entropy

[0078]

[0079] in |D f | is the number of instances in the data set (i.e., the feature component f i The number of ij is the value of the characteristic component, |a ij | is the characteristic component f in each eigenvector i The value is a ij The number of .

[0080] Mutual Information

[0081] Mutual Information is a measure of the degree of mutual dependence between random variables, that is, the amount of information contained in one variable about another variable. Its definition formula is

[0082]

[0083] Where p(x, y) is the joint probability distribution of random variables X and Y, and p(x) and p(y) are the marginal probability distributions of random variables X and Y, respectively.

[0084] In the present invention, each characteristic component f is calculated i With other characteristic components fj Mutual Information of (i≠j)

[0085]

[0086] The joint probability distribution |D ij | is the characteristic component f i The number of eigenvalues ​​f j The number of ik , a jl )| is the characteristic component f i The median value is a ik The number and characteristic components f j The median value is a jl The marginal probability distribution is What is obtained here is the mutual information between each feature component. In order to obtain the mutual information of a single feature component relative to all feature components, the average normalized mutual information is used here

[0087]

[0088] Where I(f i ;f j ) is the characteristic component f i and f j The mutual information between i ) and H(f j ) are the characteristic components f i and f j The information entropy of . The average normalized mutual information reflects the characteristic component f i The overall mutual information size between it and all other feature components.

[0089] Weight matrix

[0090] According to the amount of information within and between feature components reflected by information entropy and average normalized mutual information, the weight of feature components can be obtained by subtracting information entropy from average normalized mutual information, that is, using the formula

[0091] W(f i )=H(f i )-AVGMI NORM (f i ), (6)

[0092] Where H(f i ) is the characteristic component f i Information entropy, AVGMI NORM (f i ) is the characteristic component f i The average normalized mutual information of .

[0093] After obtaining the feature component weights of each scene according to Formula 6, a 48555-dimensional feature weight column vector can be generated for each scene. The feature weight column vectors are horizontally spliced ​​and all 0 rows are deleted (all 0 rows mean that the feature component has no contribution to classification in each scene) to obtain the weight matrix W 权重向量维数×4 At this time, each column of the feature weight matrix is ​​the importance information of each scene for each feature component, and each row is the importance of the same feature component in each scene.

[0094] Step 3: Use support vector machine and weight matrix regularization to unify the weights of each scene

[0095] Support Vector Machine

[0096] Support Vector Machines (SVM) is a supervised learning algorithm and one of the most influential machine learning algorithms. It can perform linear classification as well as nonlinear classification through kernel methods. The loss function is the most important part of the support vector machine and even all machine learning algorithms. It reflects the gap between the model and the data. The setting of the loss function directly affects the quality of the model training effect. The more commonly used loss functions include 0-1 loss function, square loss function, absolute value loss function, hinge loss function, etc. The most commonly used loss function in support vector machines is the hinge loss function, and its mathematical expression is

[0097]

[0098] in is the predicted value of the sample, y∈{-1,1} is the true value of the sample, and the max() function takes 0 and The maximum value of, that is, the loss function is 0 when the classification is correct, otherwise it is

[0099] The present invention uses a support vector machine as a classifier in the weight matrix regularization process.

[0100] Weight matrix regularization

[0101] The purpose of applying regularization in machine learning is to impose certain restrictions on the model, which can prevent the model from overfitting.

[0102] The present invention applies a weight matrix to the objective function. 2，1 Norm regularization. 2，1 The definition formula of norm is

[0103]

[0104] where X is an r-row and t-column matrix, x ij is the i-th row and j-th column element of matrix X.

[0105] From the formula we can see that l 2，1 The norm is the l1 norm of the l2 norm of each row of the matrix. 2，1 The norm ensures the row sparsity of the matrix while ensuring the density of each row and column of the matrix. For the weight matrix used in the present invention, ensuring the column density of each row ensures the convergence of the weights of the same feature component in the four scenes, and ensuring the column sparsity ensures that the weights of the feature components important to the four tasks are increased, while the weights of the unimportant ones are reduced. Therefore, the objective function used in the present invention is

[0106]

[0107] Get the target weight matrix, where u is the number of scenes, w is the number of users, loss(·) is the loss function, α is the regularization coefficient, and w i is the feature weight of the i-th scene, is the feature vector of the jth user in the i-th scene, is the label of the jth user in the i-th scene, |||| 2， is the feature weight moment W 2，1 Norm.

[0108] The present invention uses the hinge loss function and determines the learning rate and regularization coefficient through the following steps. First, the regularization coefficient is set to 0, and 10 numbers are taken as the learning rate in four value ranges [0.0001, 0.001), [0.001, 0.01), [0.01, 0.1), [0.1, 1) at equal intervals. These learning rates are used for learning, and the learning rate is iterated respectively. After that, the loss values ​​are obtained and applied to draw the loss value curve. The learning rate corresponding to the lowest point of the curve is selected as the learning rate for application. After determining the learning rate, the regularization coefficient is adjusted according to the test set accuracy (Validation Accuracy). First, the order of magnitude of the regularization coefficient is determined, and the regularization coefficient is set to 1. The regularization coefficient is changed according to the test set accuracy (if the test set accuracy is lower than the training set accuracy, the regularization coefficient is increased by 10 times, and if the test set accuracy is higher than the training set accuracy, the regularization coefficient is reduced by 10 times). A regularization coefficient order of magnitude with good test set accuracy and training set accuracy is obtained, and the regularization coefficient is increased at this order of magnitude (for example, if the optimal regularization coefficient order of magnitude is 0.1, then 0.2, 0.3, etc. are selected as the regularization coefficient at the same order of magnitude). The coefficient with the best test set accuracy performance is selected as the final regularization coefficient α.

[0109] After determining the learning rate and regularization coefficient, the objective function is iterated for four scenarios at most. times, and the model accuracy no longer improves so that the objective function converges to obtain l 2，1 The weight matrix after norm regularization.

[0110] Step 4: Select features based on the weight matrix and use the long short-term memory artificial neural network for user identification

[0111] Feature selection

[0112] Take the average value of each row in the feature weight matrix and transpose it to obtain the multi-task average feature weight row vector

[0113] AVGW 1×权重向量维数(d) =[w 0 ,...,w d ], the feature selection method uses the formula

[0114]

[0115] where w i ,w j is the component of the average feature weight vector AVGW, threshold is the threshold, is the sum of all feature component weights, It is the sum of the weights of the m largest feature components, m starts from 0 and increases by 1 each time until the ratio is greater than the threshold, that is, the m largest feature components are selected as the input of the long short-term memory artificial neural network. Here, the threshold is set to 0.95 to obtain the m largest feature components.

[0116] Long short-term memory artificial neural network

[0117] Recurrent Neural Network (RNN) is a type of neural network model that takes sequence data as input. The hidden layer information S at a certain moment is t Not only depends on the current input X t , also depends on the hidden layer information S at the previous moment t-1 , so RNN is very suitable for sequence features that conform to time order and logical order.

[0118] The characteristics of RNN's current input combined with the previous hidden layer information will cause gradient vanishing and gradient exploding problems, which makes it unsuitable for long-term dependency problems. However, the Long Short-Term Memory (LSTM) artificial neural network can solve this problem well. It uses input gates, output gates and forget gates to select information. Figure 4 As shown, the current input and the previous output value in LSTM are concatenated and multiplied by the input gate, output gate, and forget gate weight vector W respectively. i , Wo , W f , and then pass through the activation function Sigmoid (output [0,1]) to obtain the corresponding control signal, and the spliced ​​information passes through the activation function tanh to obtain the input data to be processed At this moment, the forget gate control signal f t and the hidden layer information C at the previous moment t-1 The multiplication result and the input gate control signal i t With input data to be processed The multiplication results are added to obtain the hidden layer information at the current moment. The hidden layer information at the current moment is activated by the tanh function (output [-1, 1]) and combined with the output gate control signal O t Multiply to get the current output signal h t Through the above steps, LSTM realizes information selection and solves the gradient vanishing and gradient exploding problems of ordinary RNN.

[0119] The keystroke and mouse dynamics feature sequences collected by time belong to the time series feature sequence. There is a certain relationship between the features of the previous and next inputs. The data characteristics are consistent with the long short-term memory artificial neural network. Therefore, the present invention uses the LSTM learning and screened keystroke and mouse features to obtain a classification model suitable for four scenes and free scenes. The trained model can realize multi-scene subject recognition of keystroke dynamics and mouse dynamics.

Claims

1. A multi-scene subject recognition method based on keystroke dynamics and mouse dynamics, characterized in that: include: Step 1: Extract features and preprocess data Extract the timing features of the keyboard and mouse according to the length of the time slice; there are 5 types of keyboard timing features, namely the time interval DT between pressing and releasing a single key, the time interval UD between the release of the first key and the pressing of the second key, the time interval DD between the pressing of the first key and the pressing of the second key, the time interval UU between the release of the first key and the release of the second key, and the time interval DU between the pressing of the first key and the release of the second key; The keystroke feature is applicable to each key and each key pair. For a full-size 110-key keyboard, the DT feature of a single key in each time slice is 110 dimensions in total, and the UD, DD, UU, and DU features between key pairs are 12100 dimensions each, totaling 48400 dimensions, so the keystroke feature in each time slice is 48510 dimensions in total; there are 7 types of mouse timing features, namely, the average moving speed MDS of each distance divided by 100 pixels, the average moving speed MDA of each direction, the proportion of operations in each direction MDH, the average moving speed ATA of each operation type, the proportion of operations of each operation type ATH, the proportion of operations of each moving distance TDH, and the proportion of operations in each time MTH; among them, the average moving speed MDS of each distance, the average moving speed MDA of each direction, the proportion of operations of each direction MDH, and the proportion of operations of each moving distance TDH are defined according to 8 directions and every 100 pixel spacing within 1 to 800 pixels, so they are 8-dimensional features; The two features, average moving speed ATA of each operation type and operation quantity ratio ATH of each operation type, are defined based on the four operation types of users: moving, dragging, prohibiting, and clicking. Therefore, four-dimensional features are extracted from the four operations. The proportion of the number of operations in each time period MTH is a 200-millisecond time interval within 1 to 1000 milliseconds, so it is a 5-dimensional feature; finally, the mouse features in each time slice are 45 dimensions in total, and the feature vector obtained in each time slice has 45 elements; the keystroke features and mouse features are obtained separately and then spliced ​​according to the time slice to obtain a 48555-dimensional feature vector, and the corresponding feature components that do not appear in a single time slice are filled with 0; The eigenvectors with all 0 eigencomponents in a single time slice and the eigencomponents with all 0 in all eigenvectors are deleted, and the data preprocessing is completed using the homogeneous mean interpolation method; Step 2: Use information entropy and mutual information methods to generate feature weight vectors for each scene, and merge the weight vectors into a weight matrix The eigenvector is defined as f = [f1,f2,f3,f4,...,f n-1 ,f n ], where each f i is a characteristic component, and n can be any number; Information Entropy is a quantitative measure of information, that is, the expected amount of information about an event. Its definition formula is Where p(x i ) is a random event x i Probability of occurrence; Based on the above information entropy definition formula, each feature component f is calculated i Information entropy in |D f | is the number of instances in the data set, i.e., the feature component f i The number of ij is the value of the characteristic component, |a ij | is the characteristic component f in each eigenvector i The value is a ij The number of The mutual information definition formula is: Where p(x,y) is the joint probability distribution of random variables X and Y, p(x) and p(y) are the marginal probability distributions of random variables X and Y respectively; Calculate each eigenvalue f i With other characteristic components f j Mutual information of i≠j The joint probability distribution |D ij | is the characteristic component f i The number of eigenvalues ​​f j The number of ik ,a jl )| is the characteristic component f i The median value is a ik The number and characteristic components f j The median value is a jl The product of the number of ; marginal probability distribution What is obtained here is the mutual information between each feature component. In order to obtain the mutual information of a single feature component relative to all feature components, the average normalized mutual information is used here Where I(f i ;f j ) is the characteristic component f i and f j The mutual information between i ) and H(f j ) are the characteristic components f i and f j The information entropy of ; the average normalized mutual information reflects the characteristic component f i The overall mutual information size with all other feature components; According to the amount of information within and between feature components reflected by information entropy and average normalized mutual information, the feature component weight is obtained by subtracting information entropy from average normalized mutual information, that is, using the formula W(f i )=H(f i )-AVGMI NORM (f i ), (6) Where H(f i ) is the characteristic component f i Information entropy, AVGMI NORM (f i ) is the characteristic component f i The average normalized mutual information of After obtaining the feature component weights of each scene according to formula (6), a 48555-dimensional feature weight column vector is generated for each scene. The feature weight column vectors are horizontally spliced ​​and all zero rows are deleted to obtain the weight matrix W 权重向量维数×4 , all 0 rows mean that this feature component has no contribution to classification in each scene; At this time, each column of the feature weight matrix is ​​the importance information of each scene for each feature component, and each row is the importance of the same feature component in each scene; Step 3: Use support vector machine and weight matrix regularization to unify the weights of each scene The hinge loss function in the support vector machine is expressed as in is the predicted value of the sample, y∈{-1,1} is the true value of the sample, and the max() function takes 0 and The maximum value of, that is, the loss function is 0 when the classification is correct, otherwise it is Use support vector machines as classifiers in the weight matrix regularization process; Apply the weight matrix l to the objective function 2,1 Norm regularization; l 2,1 The definition formula of norm is where X is an r-row and t-column matrix, x ij is the i-th row and j-th column element of matrix X; See l through the formula 2,1 The norm is the l1 norm of the l2 norm of each row of the matrix; l 2,1 The norm ensures the row sparsity of the matrix while ensuring the density of each row and column of the matrix. For the weight matrix used, ensuring the column density of each row ensures the convergence of the weights of the same feature component in the four scenes, and ensuring the column sparsity ensures that the weights of the feature components that are important to the four tasks are increased, and the weights of the unimportant ones are reduced; therefore, the objective function is used Get the target weight matrix, where u is the number of scenes, w is the number of users, loss(·) is the loss function, α is the regularization coefficient, and w i is the feature weight of the i-th scene, is the feature vector of the jth user in the i-th scene, is the label of the jth user in the i-th scene, ||W|| 2,1 is the feature weight moment W 2,1 norm; Hinge Loss function is used, and the learning rate and regularization coefficient are determined by the following steps: first, the regularization coefficient is set to 0, and 10 numbers are taken as the learning rate in four value ranges [0.0001, 0.001), [0.001, 0.01), [0.01, 0.1), [0.1, 1) at equal intervals, and these learning rates are used for learning, and the learning rate is iterated respectively. After that, we get the loss values ​​and apply them to draw the loss value curve. We select the learning rate corresponding to the lowest point of the curve as the learning rate for application. After determining the learning rate, adjust the regularization coefficient according to the test set accuracy. First, determine the order of magnitude of the regularization coefficient, set the regularization coefficient to 1, and change the regularization coefficient according to the test set accuracy. If the test set accuracy is lower than the training set accuracy, increase the regularization coefficient by 10 times. If the test set accuracy is higher than the training set accuracy, reduce the regularization coefficient by 10 times. Finally, confirm the order of magnitude of the regularization coefficient and increase the regularization coefficient under this order of magnitude. Select the coefficient with the best test set accuracy as the final regularization coefficient α. After determining the learning rate and regularization coefficient, the objective function is iterated for four scenarios at most. times, and the model accuracy no longer improves so that the objective function converges to obtain l 2,1 The weight matrix after norm regularization; Step 4: Select features based on the weight matrix and use the long short-term memory artificial neural network for user identification Take the average value of each row in the feature weight matrix and transpose it to obtain the multi-task average feature weight row vector AVGW 1 ×权重向量维数d =[w 0 ,...,w d ], the feature selection method uses the formula where w i ,w j is the component of the average feature weight vector AVGW, threshold is the threshold, is the sum of all feature component weights, It is the sum of the weights of the m largest feature components, m starts from 0 and increases by 1 each time until the ratio is greater than the threshold, that is, the m largest feature components are selected as the input of the long short-term memory artificial neural network; here the threshold is set to 0.95 to obtain the m largest feature components; In LSTM, the current input and the previous output are concatenated and multiplied by the input gate, output gate, and forget gate weight vector W respectively. i , W o , W f , and the corresponding control signals are obtained after the activation function Sigmoid, and the spliced ​​information is activated by the tanh function to obtain the input data to be processed At this moment, the forget gate control signal f t and the hidden layer information C at the previous moment t-1 The multiplication result and the input gate control signal i t With input data to be processed The multiplication results are added to obtain the hidden layer information at the current moment. The hidden layer information at the current moment is output [-1,1] through the activation function tanh and combined with the output gate control signal O t Multiply to get the current output signal h t .

Images