User identity recognition method and system based on multi-source inertial sensor data

Abstract

The invention discloses a user identity recognition method based on multi-source inertial sensor data, and the method comprises the steps: obtaining user behavior data through user equipment, carrying out the preprocessing of the user behavior data, and generating first behavior data; segmenting the first behavior data through sliding windows, wherein each window comprises at least one complete action cycle; and for the first behavior data of each window, the user identity is identified by using the trained user identity identification model, so that continuous authentication of the user identity is realized, and in the training of the user identity identification model, preliminary feature extraction is performed by using a tsfreh method, and features are expanded by adopting a similar expansion mode. According to the method, more comprehensive feature coverage can be realized, and the recognition performance is improved.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent terminal security authentication, and in particular relates to a user identity recognition method and system based on multi-source inertial sensor data. Background Art

[0002] With the rapid development of the Internet of Things (IoT) and mobile computing technologies, smart wearable devices that integrate mobile payment, health monitoring, and identity authentication have become an indispensable part of modern life. Against this backdrop, information security issues are becoming increasingly prominent, and traditional password authentication methods are no longer able to meet current security needs. Biometric recognition technology, due to its unique individuality and anti-counterfeiting properties, has become a key solution for ensuring information security.

[0003] Common biometric features include fingerprints and facial features, and the recognition technology for these features is relatively mature. Human behavior is also a form of information expression. During the recognition process, different behaviors can reflect different behavioral patterns. Behavioral biometric recognition is a technology that uses the analysis of an individual's unique behavioral patterns to authenticate identity. Unlike physiological biometrics such as fingerprints and irises, behavioral biometrics capture dynamic behavioral habits exhibited during interaction. Its persistence and concealment not only improve the security of electronic products but also allow for application in more secure environments, providing a new approach to identity verification. Due to individual differences in height, weight, and bone structure, each person exhibits unique characteristics during movement. Therefore, effective kinematic parameters can be used to extract effective features and evaluate different individuals.

[0004] Behavioral biometric recognition based on inertial sensing offers the advantages of long-range recognition, sustainability, and non-interaction. Common feature research methods include feature-based machine learning and deep learning. However, most current methods for identity recognition based on inertial sensor data rely on deep learning. This lacks interpretability and makes it unsuitable for high-security scenarios. Furthermore, current methods only extract a small number of statistical features and fail to fully utilize the broader range of features from users' daily activities, thus limiting improvements in recognition accuracy. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a user identity recognition method based on multi-source inertial sensor data, which can extract biometric features from a variety of daily behavioral activities of users and adopt a similar expansion strategy to significantly increase the amount of effective information, thereby improving the accuracy of identity recognition.

[0006] The purpose of the present invention is achieved through the following technical solutions:

[0007] A user identification method based on multi-source inertial sensor data, comprising:

[0008] Obtain user behavior data through the user device, wherein the user behavior data is three-dimensional data;

[0009] Preprocessing the user behavior data to generate first behavior data;

[0010] The first behavior data is segmented using a sliding window, where each window includes at least one complete action cycle;

[0011] For the first behavior data of each window, the trained user identity recognition model is used to identify the user identity, thereby achieving continuous authentication of the user identity.

[0012] The training of the user identity recognition model includes:

[0013] Acquiring, through a user device, training data of a target user, wherein the training data includes daily behavior data corresponding to a plurality of daily behaviors of the target user;

[0014] For each daily behavior, perform the following steps S1-S5:

[0015] S1: Preprocessing the daily behavior data and segmenting the data using a sliding window, so that each window includes at least one complete action cycle, and each window data corresponds to the identity tag of the target user;

[0016] S2: For each window of data, the tsfresh automatic feature engineering method is used to extract preliminary features of the window data. The preliminary features are divided into positional features, variational features, and structural features. Positional features characterize the central tendency or spatial location of the data; variational features characterize the fluctuation and dispersion of the data; and structural features characterize the distribution form, internal relationship, or high-order pattern of the data.

[0017] S3: Using a feature importance evaluation method to score the importance of each preliminary feature, selecting the first preset number of preliminary features with the highest importance scores as important features, and generating an important feature set;

[0018] S4: Determine whether the important feature set includes all feature categories. If so, execute S5. If not, add a second preset number of features with the highest importance scores corresponding to feature categories not included in the important features into the important feature set as important features.

[0019] S5: Train the model based on the features in the important feature set and the labels corresponding to each window data.

[0020] Through the above training process, a user identity recognition model corresponding to each daily behavior is obtained, and when the user performs any of the multiple daily behaviors, the user's identity can be recognized.

[0021] Furthermore, the user device is a smart device such as a smart phone, a smart bracelet, etc.

[0022] Furthermore, user behavior data comes from an accelerometer or gyroscope in the user's device.

[0023] Furthermore, location features include the mean, median, maximum, minimum and sum of the data; variation features include the variance, standard deviation, average change value, etc. of the data; and structure features include the absolute maximum and skewness of the data.

[0024] The present invention also provides a user identification system based on multi-source inertial sensor data, comprising:

[0025] a memory configured to store a computer program;

[0026] The processor is configured to execute the computer program to implement the user identification method based on multi-source inertial sensor data as described above.

[0027] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method for user identification based on multi-source inertial sensor data as described above is implemented.

[0028] The beneficial effects of the present invention are:

[0029] This invention extracts biometric features from behavior for identity recognition, using a lightweight feature extraction method that is easier to apply to real-world scenarios. The extracted features, such as the median and standard deviation, have certain physical meanings and strong model visibility, making them more suitable for scenarios with higher security.

[0030] The present invention also significantly increases the amount of effective information by enriching feature types and categories and adopting similar expansion strategies, enabling the model to capture more distinguishing features, thereby improving the accuracy of user identification and verification. It can be deployed on mobile device platforms to periodically verify user identities in fixed time windows, laying the foundation for achieving imperceptible continuous identity authentication and improving system security and user experience.

[0031] Other advantages, objects, and features of the present invention will be described in part in the following description and, in part, will be apparent to those skilled in the art upon examination of the following description or may be learned from practice of the present invention. The objects and other advantages of the present invention may be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings, in which:

[0033] Figure 1 is a schematic flow chart of a method for user identification based on multi-source inertial sensor data;

[0034] Figure 2 It is a schematic flow chart for training a user identity recognition model;

[0035] Figure 3 Schematic diagram of user identification based on multi-source inertial sensor data. DETAILED DESCRIPTION

[0036] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the preferred embodiments are only for illustrating the present invention, and are not intended to limit the scope of protection of the present invention.

[0037] Combine Figure 1 and Figure 3 As shown, the user identification method based on multi-source inertial sensor data includes:

[0038] User behavior data is obtained through the user device, wherein the user behavior data is three-dimensional data; the user device can be a smart device such as a smart phone, a smart bracelet, etc., and the user behavior data comes from an accelerometer or a gyroscope in the user device.

[0039] Preprocess the user behavior data to generate first behavior data; the preprocessing may include normalizing the user behavior data. For example, the received user behavior data may be of different dimensions. In this case, the data needs to be transformed (e.g., normalized) before inputting into the model to ensure the convergence and convergence speed of the model.

[0040] In some embodiments, the data may be transformed using a standard deviation normalization method. Since the user data is multidimensional data within a continuous time range (i.e., a multidimensional time series), the following formula may be performed for each dimensional time series:

[0041]

[0042] Where X={x1,...,x i ,...,x n} represents a time series of any dimension, x i Represents the data in the time series, x i ′ is x i The normalized results constitute the first line of data, where μ is the mean of the data in the time series X, and σ is the standard deviation of the data in the time series X.

[0043] Next, the first behavior data is segmented through a sliding window. Since the amplitude and frequency of each person's movements during the exercise process are different, the window can be set so that each window includes at least one complete movement cycle. For example, for the daily behavior of climbing stairs, the time it takes to climb at least one step can be used as a length. In this way, the data in this window will contain the complete action of climbing stairs, thereby providing a basis for subsequent identity recognition when climbing stairs. If the length of a window is less than the time it takes to climb a step, then a window does not contain a complete stair climbing data. For example, if the user just lifts his foot and the window time ends, it is impossible to determine whether the foot-lifting action is climbing stairs or kicking, and the user's identity cannot be recognized based on the leg-lifting action.

[0044] The window size can be set to 10 seconds, and the number of data points in a window can be obtained according to the data collection frequency. In some embodiments, the data can also be segmented by sliding the window according to a certain overlap ratio (for example, an overlap ratio of 50%).

[0045] After the data segmentation is completed, the first line of data in each window is used to identify the user identity using the trained user identity recognition model, thereby achieving continuous authentication of the user identity.

[0046] Figure 2 This is a schematic flow chart for training the user identity recognition model, combined with Figure 2 , training the user identity recognition model includes:

[0047] The training data of the target user is obtained through the user device, wherein the training data includes daily behavior data corresponding to various daily behaviors of the target user; wherein the target user is a daily user of the user device, and the target user's daily behaviors include: running (jogging), going up and down stairs, sitting, standing, typing, writing, washing face, brushing teeth, eating, eating snacks, drinking water, kicking, bouncing a ball, clapping, folding clothes, etc. 18 types.

[0048] For each daily behavior, perform the following steps S1-S5:

[0049] S1: Preprocessing the daily behavior data corresponding to the daily behavior and segmenting the data using a sliding window, so that each window includes at least one complete action cycle, and each window data corresponds to the identity tag of the target user;

[0050] S2: For each window of data (i.e., window data), extract preliminary features of the window data using the tsfresh automatic feature engineering method, where the preliminary features constitute a minimum feature set. In some embodiments, data of different dimensions can be extracted using parallel computing to accelerate the feature extraction process.

[0051] Before using the tsfresh automatic feature engineering method to extract features, the window data needs to be converted into the long format required by tsfresh, which includes fields such as window ID, time point, and sensor value.

[0052] Features extracted by the tsfresh method include statistical features such as the mean, median, maximum, minimum, standard deviation, variance, and absolute sum of the data. They can also include time-series related features such as the autocorrelation coefficient of different lag orders, the slope and intercept of linear regression, and frequency domain spectral analysis features such as the Fourier coefficients of different frequencies obtained using Fourier transform, power spectrum density, peak frequency, and spectrum entropy. They can also include other features such as nonlinear features and distribution features. It should be noted that tsfresh can extract many types of features, and then its built-in feature selection method can be used to select some of them to form a minimum feature set (i.e., preliminary features).

[0053] The above preliminary features can be divided into location features, variation features, and structural features. Location features characterize the central tendency or spatial location of the data (for example, the mean, median, maximum, minimum, sum, coordinates of data points, etc.); variation features characterize the fluctuation and dispersion of the data (for example, the variance, standard deviation, range (i.e., the difference between the maximum and minimum values ​​of the data), average change (i.e., the average value of the change in the data over a period of time), etc.); structural features characterize the distribution form, internal relationships, or high-order patterns of the data (such as the absolute maximum and absolute minimum values ​​of the data, kurtosis, skewness, etc.).

[0054] In some embodiments, after completing preliminary feature extraction, feature post-processing can be performed, such as screening for significant features through hypothesis testing, ultimately outputting a feature matrix that can be used in machine learning models. This post-processing process can efficiently extract discriminative time series features from raw sensor data, providing effective feature representation for subsequent tasks such as user behavior recognition or identity authentication.

[0055] S3: Use a feature importance assessment method (e.g., the feature importance assessment mechanism built into the random forest algorithm) to score the importance of each preliminary feature, select the first preset number (which can be any appropriate value, for example, the top 6, top 10, etc.) of preliminary features with the highest importance scores as important features, and generate an important feature set. For example, the top six preliminary features with the highest importance scores can be selected as important features. The top six preliminary features may include all feature categories, i.e., positional features, variational features, and structural features, or may only include one or two of the feature categories. Therefore, proceed to step S4.

[0056] S4: Determine whether all feature categories are included in the important feature set. If the important feature set obtained in S3 already contains all feature types, execute S5; if not all feature categories are included, add the second preset number (which can be any appropriate value) of features with the highest importance scores corresponding to the feature categories not included in the important feature set as important features to the important feature set (i.e. Figure 2 For example, if the important feature set generated in S3 only includes position features and variation features but lacks structure features, it is necessary to obtain the importance score results of the structure features in S3 and add the second preset number (for example, the top 2, top 3, top 4, or any other appropriate number) of structure features with the highest importance score results to the important feature set.

[0057] After obtaining the important feature set through feature importance evaluation, the present invention can achieve more comprehensive feature coverage and improve recognition performance by introducing features with high importance rankings of feature categories that are not included.

[0058] S5: Train the model based on the features in the important feature set (i.e., all the features in the important feature set finally obtained in S4) and the labels corresponding to each window data.

[0059] In this invention, a five-fold crossover is also used to test the generalization performance of the model. Finally, the accuracy is used as an evaluation index to test the generalization performance of the model. The larger the value of each index, the better the generalization of the model.

[0060] Through the above training process, a user identity recognition model corresponding to each daily behavior is obtained, and when the user performs any of the multiple daily behaviors, the user's identity can be recognized.

[0061] The user identity recognition model can be stored in the user device or in a remote server to provide identity recognition services for various applications. For example, for a smart phone, when the target user carries the smart phone to complete daily behavioral activities, the user identity recognition model can be trained by collecting the target user's daily behavioral data. When a person (not necessarily the target user himself) carries the smart phone, the accelerometer and / or gyroscope in the smart phone obtains the behavior data of this person, and the trained user identity recognition model is used to identify this person, that is, it can be determined whether this person is the target user. If it is the target user, the user device can accept the person's operation on the device. If it is not the target user, the user device can reject the person's operation on the device. In some cases, the user device can automatically trigger an alarm or send reminder information to other devices associated with the user device.

[0062] The present invention can achieve identity authentication of the data in each window (for example, 10 seconds) by processing the data in each window, that is, continuous identity authentication. For example, for smartphones, tablets, etc., continuous authentication using posture, gait, touch habits, etc. can ensure that the device user is the owner himself; for social and communication software, it can prevent others from impersonating and chatting, defrauding, etc. after the account is stolen. Therefore, the present invention periodically verifies the user's identity in a fixed time window, laying the foundation for realizing imperceptible continuous identity authentication, thereby improving system security and user experience.

[0063] The present invention also provides a user identification system based on multi-source inertial sensor data, comprising:

[0064] a memory configured to store a computer program;

[0065] The processor is configured to execute the computer program to implement the user identification method based on multi-source inertial sensor data as described above.

[0066] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method for user identification based on multi-source inertial sensor data as described above is implemented.

[0067] The advantages of the present invention are described below through specific experiments.

[0068] This experiment uses the public dataset WISDM2019, collected by the WISDM Laboratory at Fordham University in the United States. It contains behavioral data from 51 volunteers performing 18 daily activities, such as walking, running, and climbing stairs. Each participant wore a smartwatch (LG G Watch) and a smartphone (Google Nexus 5 / 5X or Samsung Galaxy S5) in their pocket. A dedicated app simultaneously recorded triaxial motion data from the accelerometer and gyroscope at a sampling rate of 20Hz. Each activity lasted 3 minutes, and each participant contributed a total of 54 minutes of data. This dataset was collected in real-life settings, rather than in a laboratory environment, and thus has high practical value. The data in this experiment comes from 51 volunteers, all of whom showed normal walking performance. After data collection, each volunteer was labeled for identification. After data preprocessing and feature selection, a five-fold cross-validation was performed. The data was randomly divided into five groups, four for training and one for testing. Feature selection was performed only on the test set. Identity identification was then performed on the validation set based on these selected features. The final experimental results are shown in Table 1 below:

[0069] Table 1 Experimental results

[0070]

[0071] Table 1 shows the experimental results of the recognition accuracy (%) of 18 types of daily activity behavior data of users collected using the accelerometer of a mobile phone, the gyroscope of a mobile phone, the accelerometer of a watch (i.e., a bracelet), and the gyroscope of a watch. The result in the last row is the average of the 18 rows above.

[0072] The results in Table 1 demonstrate the good generalization performance of lightweight features in behavioral biometrics. Mobile phone acceleration data performs best for identity verification, achieving an average recognition accuracy of 96.73% across 18 motions. Different modalities of data from the same device were also used for identity verification. Acceleration-based identification achieved significantly higher accuracy than gyroscope-based identification in both devices. Therefore, among single-modality devices, three-dimensional acceleration signals offer the best recognition performance. Periodic or large-scale movements (such as walking, jogging, clapping, catching a ball, and dribbling) exhibit high recognition accuracy (generally exceeding 95%). However, accuracy is lower for static activities (such as sitting and standing) or fine movements (such as brushing teeth, eating chips, and folding clothes). In health monitoring or behavioral analysis scenarios, mobile phone sensors are more suitable for tasks requiring high precision (such as gait analysis), while watches should be used with caution for static or localized motion recognition. For dietary or fine movement monitoring, combining other modalities (such as audio or vision) may be necessary to improve reliability.

[0073] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions, which should all be included in the scope of the claims of the present invention.

Claims

1. A user identification method based on multi-source inertial sensor data, characterized in that: include: Obtain user behavior data through the user device, wherein the user behavior data is three-dimensional data; Preprocessing the user behavior data to generate first behavior data; The first behavior data is segmented using a sliding window, where each window includes at least one complete action cycle; For the first behavior data of each window, the trained user identity recognition model is used to identify the user identity, thereby achieving continuous authentication of the user identity. The training of the user identity recognition model includes: Acquiring, through a user device, training data of a target user, wherein the training data includes daily behavior data corresponding to a plurality of daily behaviors of the target user; For each daily behavior, perform the following steps S1-S5: S1: Preprocessing the daily behavior data corresponding to the daily behavior and segmenting the data using a sliding window, so that each window includes at least one complete action cycle, and each window data corresponds to the identity tag of the target user; S2: For each window of data, the tsfresh automatic feature engineering method is used to extract preliminary features of the window data. The preliminary features are divided into positional features, variational features, and structural features. Positional features characterize the central tendency or spatial location of the data; variational features characterize the fluctuation and dispersion of the data; and structural features characterize the distribution form, internal relationship, or high-order pattern of the data. S3: Using a feature importance evaluation method to score the importance of each preliminary feature, selecting the first preset number of preliminary features with the highest importance scores as important features, and generating an important feature set; S4: Determine whether the important feature set includes all feature categories. If all feature categories are included, execute S5. If not all feature categories are included, add a second preset number of features with the highest importance scores corresponding to feature categories not included in the important features as important features to the important feature set. S5: Train the model based on the features in the important feature set and the labels corresponding to each window data. Through the above training process, a user identity recognition model corresponding to each daily behavior is obtained, and when the user performs any of the multiple daily behaviors, the user's identity can be recognized.

2. The user identification method based on multi-source inertial sensor data according to claim 1, characterized in that: The user device is a smart device such as a smart phone or a smart bracelet.

3. The user identification method based on multi-source inertial sensor data according to claim 1, characterized in that: User behavior data comes from the accelerometer or gyroscope in the user's device.

4. The user identification method based on multi-source inertial sensor data according to claim 1, characterized in that: Positional features include the mean, median, maximum, minimum, and sum of the data; variation features include the standard deviation, variance, average change, and range of the data; structural features include the absolute maximum and absolute minimum of the data, the skewness and kurtosis of the data.

5. A user identification system based on multi-source inertial sensor data, characterized in that: include: a memory configured to store a computer program; A processor is configured to execute the computer program to implement the user identification method based on multi-source inertial sensor data according to any one of claims 1 to 4.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for user identification based on multi-source inertial sensor data according to any one of claims 1 to 4 is implemented.

Images