A multi-level memory-based enhanced auto-encoder continuous identity authentication method

Abstract

The present application relates to a kind of based on multistage memory enhanced auto-encoding continuous identity authentication method, belong to continuous authentication technical field, comprising the following steps: S1: collect user behavior data;S2: user behavior data is normalized;S3: construct the data enhancement model based on VAE, and training is carried out using normalized data;S4: construct the data reconstruction model based on MulMemAE, and training is carried out using normalized data and VAE enhanced data;S5: the user behavior data that real-time collection and normalization is input into the data reconstruction model after training, to reconstruct user behavior data;S6: the reconstructed user behavior data is compared with the input user behavior data, and reconstruction error is calculated;S7: reconstruction error is compared with preset threshold, if error is lower than preset threshold, then user is legitimate user;otherwise user is imposter, and requires user to log in again.

Description

Technical Field

[0001] The present invention belongs to the technical field of continuous authentication and relates to a continuous identity authentication method based on a multi-level memory enhanced autoencoder. Background Art

[0002] In today's information age, mobile devices such as smartphones, smartwatches, and tablets have become an integral part of daily life, profoundly changing the way we communicate, access information, work, and entertain ourselves. These devices have significantly enhanced the convenience and connectivity of modern life. Smartphones enable online banking, allowing us to easily manage our finances, transfer money, and pay bills; they also facilitate email communication, providing instant access to personal and professional communications, and support mobile payments for fast and secure transactions. Smartwatches, on the other hand, focus on health monitoring, helping users maintain and improve their health by tracking metrics such as heart rate, steps, and sleep patterns. Tablets are particularly useful for remote work, supporting productivity applications, video conferencing, and collaboration tools, allowing users to work efficiently anytime, anywhere.

[0003] While mobile devices have brought significant convenience to our daily lives, they have also raised several privacy concerns. Exposure of information stored in mobile devices can lead to significant losses for users, highlighting the need for robust security mechanisms. Traditional authentication methods include knowledge-based and biometric-based methods. Knowledge-based methods, such as personal identification numbers (PINs), passwords, tokens, and graphical passwords, rely on the user's memory but are vulnerable to various attacks, including password guessing, dictionary attacks, password leakage, and man-in-the-middle attacks. Biometric-based mechanisms, such as fingerprint, facial recognition, iris, and palm vein recognition, generally provide higher security but can also be subject to forgery and shadowing attacks. These two traditional authentication methods only provide security during login and become ineffective once an attacker gains access while the device is in use. This vulnerability further emphasizes the urgent need to develop more continuous and robust protection mechanisms to safeguard personal information on mobile devices.

[0004] Continuous authentication is an implicit approach that enhances security by continuously monitoring and verifying a user's identity as they interact with a device, thereby preventing unauthorized access and attacks. This approach leverages user behavioral characteristics, such as gait, chest movement, and touch gestures, which are involuntary and typically exhibit consistent patterns. These behaviors occur naturally, requiring no conscious effort from the user, making continuous authentication both feasible and user-friendly. Continuous authentication systems typically consist of two main phases: enrollment and verification. During the enrollment phase, mobile devices capture behavioral data from authorized users through built-in sensors and use this data to train a model to recognize individual user patterns. During the verification phase, the system periodically collects sensor data to determine whether the current user is a legitimate user of the device. If the user is confirmed to be legitimate, access continues uninterrupted; otherwise, the system locks the device and requires a new login, thus ensuring the security of user information on the device.

[0005] Most research on continuous authentication employs a feature extractor-classifier architecture, where a feature extractor projects sensor data into a higher-level feature representation. A classifier then uses these features and corresponding labels to distinguish legitimate from unauthorized users. Meanwhile, reconstruction-based methods are emerging as a new type of continuous authentication approach. These methods involve a training phase, in which a model encodes and decodes raw sensor data, generates reconstructed data, and calculates the reconstruction error to continuously optimize the model. If the reconstruction error falls below a preset threshold during the verification phase, the user is deemed legitimate; otherwise, the user is deemed unauthorized.

[0006] The feature extractor-classifier paradigm requires separate training of the feature extractor and classifier, increasing the complexity of the overall training process. Insufficient training of either component can compromise verification performance. Furthermore, reconstruction-based architectures are still under development, and the reconstruction process struggles to effectively distinguish between normal and abnormal samples. Both feature extractor-classifier and reconstruction-based approaches require large amounts of user data to ensure effective training. To alleviate data scarcity, researchers have employed techniques such as permutation, scaling, flipping, sampling, dithering, and cropping, similar to image enhancement methods. However, these techniques may not be fully adapted to sensor data and often result in homogenized training samples. Consequently, some researchers have turned to deep learning for data augmentation, primarily employing generative adversarial networks (GANs). While GANs simultaneously train the generator and discriminator models, generating more diverse data, they can also lead to unstable training processes, such as mode collapse or vanishing gradients. Furthermore, GANs typically require long training times, potentially impacting the real-time performance of user authentication. Summary of the Invention

[0007] In view of this, the object of the present invention is to provide a continuous identity authentication method based on a multi-level memory enhanced autoencoder.

[0008] In order to achieve the above object, the present invention provides the following technical solutions:

[0009] A continuous identity authentication method based on a multi-level memory-enhanced autoencoder includes the following steps:

[0010] S1: Use motion sensors on mobile devices to collect user behavior data;

[0011] S2: Normalize the collected user behavior data;

[0012] S3: Build a data augmentation model based on variational autoencoder (VAE) and train it using the data from step S2.

[0013] S4: Build a data reconstruction model based on the multi-level memory-enhanced autoencoder MulMemAE, and use the data in step S2 and the VAE-enhanced data for training;

[0014] S5: Input the real-time collected and normalized user behavior data into the trained data reconstruction model to reconstruct the user behavior data;

[0015] S6: Compare the reconstructed user behavior data with the input user behavior data and calculate the reconstruction error;

[0016] S7: Compare the reconstruction error with a preset threshold. If the error is lower than the preset threshold, the user is a legitimate user; otherwise, the user is an imposter and is required to log in again for identity verification.

[0017] Furthermore, the motion sensor in step S1 includes an accelerometer and a gyroscope, and n motion sensor samples are collected within a time period t, where n = t × f, and each sample consists of a six-dimensional vector where x, y, and z represent the three axes of each motion sensor, and a and g represent the accelerometer and gyroscope, respectively.

[0018] Furthermore, in step S2, each set of sensor data is represented as a d×n matrix; for the accelerometer, the matrix D a The structure is:

[0019]

[0020] Where d = 3 and n = t × f;

[0021] The data of the accelerometer and gyroscope are expressed as D = [D a , D g ];

[0022] Each axis of each sensor data is normalized to [0, 1] by the following formula:

[0023]

[0024] Where i = x, y, z represents the three axes of each sensor, j = 1, 2, ..., n represents each sample, and s = a, g represents the accelerometer and gyroscope respectively;

[0025] Normalized accelerometer data D anorm Expressed as

[0026] The normalized data from the accelerometer and gyroscope are compiled into a single matrix D norm =[d anorm , d gnorm ] T .

[0027] Furthermore, the data augmentation model based on variational autoencoder (VAE) includes an encoder, a latent space, and a decoder;

[0028] The encoder is divided into two branches. The first branch encodes features using four consecutive convolutional blocks, which are then concatenated with the output of the second branch. Each convolutional block contains a convolutional layer, a batch normalization, and a leaky rectified linear unit (LeakyReLU) activation function. The second branch first reshapes the input data and then processes it through a fully connected network. After concatenating the output of the second branch with the output of the first branch, two fully connected layers are used to optimize the encoding dimension.

[0029] The latent space is composed of a mean vector and a standard deviation vector, and the latent variables are obtained through the reparameterization process. z , and then passed to the decoder for reconstruction;

[0030] The decoder includes a fully connected layer, a ReLU activation function, a fully connected layer and a Sigmoid activation function connected in sequence, and finally outputs enhanced data.

[0031] Furthermore, the data reconstruction model includes an LSTM time series feature extractor, an encoder, a multi-level feature memory module and a decoder;

[0032] The LSTM time series feature extractor consists of a single-layer LSTM network. The input data is first transformed, then the LSTM network is used to capture the time series information, and finally the data is transformed through the inverse operation.

[0033] The encoder is used to convert the temporal features from the LSTM temporal feature extractor into a latent representation. The encoder consists of four layers, starting with two consecutive Convolution + Batch Normalization + ReLU (Conv+BN+ReLU) blocks. This is followed by three downsampling blocks, each of which includes a convolution layer with a stride of 2 and two Convolution + Batch Normalization + ReLU blocks. Each downsampling block performs a downsampling operation, generating encoded features of different resolutions at each step.

[0034] The decoder's structure is symmetrical to the encoder, consisting of three upsampling blocks and a convolutional layer. Each upsampling block receives input from two sources: 1) the corresponding skip connection from the encoder, and 2) upsampled features from a lower layer. These inputs are concatenated and processed through a deconvolutional layer (ConvTranspose2d), followed by two convolutional, batch normalization, and ReLU blocks and a memory module. The final convolutional layer reshapes the reconstructed data back to its original form, completing the reconstruction process.

[0035] The multi-level feature memory module is used to store the encoded feature representations from different levels of the encoder; each level of the feature memory module consists of a matrix containing N real-valued vectors, each vector has a dimension of C, expressed as Each row m of the matrix i represents a single memory item; in the initial training phase, each memory item is initialized with a random value; the training data is first processed by the LSTM temporal information extractor and then passed to the encoder to generate the encoded feature ef; the similarity between ef and each memory item is calculated by the following formula:

[0036]

[0037] The Softmax function is then used to determine the weight of each item using the following formula:

[0038] w i =Softmax(Similarity(ef,m i ))

[0039] Use sparse addressing to refine and adjust weight distribution:

[0040]

[0041] Among them, ∈ represents a very small positive number, as a threshold;

[0042] After reduction, use the following formula to Perform normalization:

[0043]

[0044] The stored latent feature lf retrieved from memory is expressed as:

[0045]

[0046] Furthermore, the reconstruction error is calculated using the mean square error (MSE), as shown in the following formula:

[0047]

[0048] Where X represents the input sensor data sequence, represents the data sequence reconstructed by MulMemAE, and i, j, and k represent the data dimensions.

[0049] Furthermore, the preset threshold is set to achieve an optimal balance between the false positive rate (FAR) and the false negative rate (FRR).

[0050] The beneficial effects of the present invention are:

[0051] 1) This paper proposes a data augmentation method based on VAE that integrates a convolutional neural network (CNN) and a fully connected network. This method enhances training stability while reducing the time required to generate augmented data, making the training process more efficient.

[0052] 2) This paper adopts a single-stage reconstruction-based authentication method. This method combines feature extraction and classification into one step, which not only simplifies the system but also improves the overall efficiency of the model.

[0053] 3) This paper uses MulMemAE to capture spatial and temporal features through multi-level feature memory. Skip connections between the encoder and decoder ensure that key high-level features are preserved, thereby improving the model's ability to distinguish legitimate data from imposter data.

[0054] Other advantages, objects, and features of the present invention will be described in part in the following description and, in part, will be apparent to those skilled in the art upon examination of the following description or may be learned from practice of the present invention. The objects and other advantages of the present invention may be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below with reference to the accompanying drawings, in which:

[0056] Figure 1This is the AuthEncoder architecture diagram;

[0057] Figure 2 This is the architecture diagram of VAE;

[0058] Figure 3 This is the structural diagram of MulMemAE;

[0059] Figure 4 (a)-(c) are the EER, accuracy and F1Score graphs with and without VAE under different data amounts;

[0060] Figure 5 (a)-(c) are the EER, Accuracy and F1Sco under different data scales and different data enhancement methods. re picture;

[0061] Figure 6 (a)-(c) are the EER, accuracy and F1 Score of AuthEncoder using MulMemAE, AE and SinMemAE on different unseen users respectively; DETAILED DESCRIPTION

[0062] The following describes the embodiments of the present invention by means of specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0063] It should be noted that the illustrations provided in the following embodiments are merely schematic illustrations of the basic concept of the present invention. Therefore, the illustrations only show components related to the present invention and are not drawn according to the number, shape, and size of components in actual implementation. In actual implementation, the type, quantity, and proportion of each component may be changed arbitrarily, and the component layout may also be more complex.

[0064] In the following description, numerous details are discussed to provide a more thorough explanation of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the embodiments of the present invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring the embodiments of the present invention.

[0065] This paper provides a method for continuous identity authentication based on a multi-level memory-augmented autoencoder. The resulting system, called the reconstruction-based continuous identity authentication system AuthEncoder, uses a variational autoencoder (VAE) for training data augmentation and a multi-level memory-augmented autoencoder (MulMemAE) for data reconstruction. AuthEncoder consists of four modules: data collection and preprocessing, VAE-based enhancement, data reconstruction, and identity authentication. The AuthEncoder's operational process includes a registration phase and a verification phase.

[0066] During the registration phase, AuthEncoder leverages the accelerometer and gyroscope embedded in mobile devices to capture user behavior patterns. The collected sensor data is normalized and used to train a VAE for data augmentation, which is then integrated with a convolutional neural network (CNN) and a fully connected network. Subsequently, the original data and the VAE-enhanced data are used to train a MulMemAE for data reconstruction. MulMemAE introduces skip connections between the encoder and decoder layers.

[0067] During the authentication phase, as users interact with their mobile devices, sensor data is collected and fed into the trained MulMemAE to reconstruct the corresponding user data. The reconstruction error is calculated by comparing the reconstructed data with the current input. Based on the reconstruction error, the AuthEncoder identifies the current user by comparing it with a preset threshold. If the error is below the threshold, the system identifies the user as legitimate; if the error exceeds the threshold, the user is considered an imposter and is required to log in again.

[0068] AuthEncoder consists of two components: a device client and a remote server. The device client is responsible for data collection and authentication, while the remote server trains the VAE and MulMemAE models. This division of labor ensures efficient client-side data processing during authentication while offloading the resource-intensive model training tasks to the server. This allows mobile devices to avoid complex computational burdens, optimizing performance and improving the overall user experience.

[0069] The AuthEncoder architecture consists of four modules: data collection and preprocessing, data enhancement based on VAE, data reconstruction based on MulMemAE, and user authentication. Figure 1As shown in the figure, in the data collection and preprocessing module, the accelerometer and gyroscope embedded in the mobile device collect user behavior data. During the registration phase, the collected data is normalized and securely transmitted to a remote server. On the server, the VAE model is trained using the transmitted data to achieve data augmentation, thereby improving the robustness of the system. During the authentication phase, the data is processed locally without being transmitted over the network.

[0070] In the data reconstruction module, MulMemAE is trained using both the original data and the VAE-augmented data. This training optimizes the reconstruction error and the entropy loss in the memory term weights to guide parameter updates within MulMemAE. After training, the trained MulMemAE parameters are returned to the mobile device for user authentication. It's important to note that both augmented and original data are used during MulMemAE training, while only original data is used during testing.

[0071] In the user authentication module, once a user interacts with a device, the system collects and normalizes sensor data in real time. This data is processed by a trained MulMemAE model, which consists of a long short-term memory (LSTM) network to capture temporal information, an encoder, multi-stage memory, and a decoder to reconstruct the input. The system then compares the reconstruction error between the current input and the reconstructed data. If the error is below a predefined threshold, the user is deemed legitimate; otherwise, the user is classified as an imposter and is required to log in again for authentication.

[0072] 1. Data collection and preprocessing

[0073] Data is collected from embedded sensors in mobile devices, including accelerometers and gyroscopes, to capture user behavior patterns. Specifically, accelerometers measure the user's coarse behavioral movements, while gyroscopes capture finer user movements. For the dataset, accelerometer and gyroscope readings were collected from 100 participants (53 males and 47 females) with a sampling rate of f = 100 Hz, and 100 minutes of data were collected for each participant. When a user interacts with the device, AuthEncoder collects n sensor data samples within a time period t, where n = t × f. Each sample consists of a six-dimensional vector where x, y, and z represent the three axes of each sensor, and a and g represent the accelerometer and gyroscope, respectively.

[0074] In the data preprocessing module, n raw data samples are collected from the accelerometer and gyroscope within a given time period t. Each set of sensor data can be represented as a d×n matrix. For the accelerometer, the matrix D a The structure is

[0075] Where d = 3 and n = t × f. The data of the accelerometer and gyroscope can be expressed as D = [D a , D g To prepare the data for training VAE and MulMemAE models, each axis of each sensor data is normalized to (0, 1] by the following formula: Where i = x, y, z represents the three axes of each sensor, j = 1, 2, ..., n represents each sample, and s = a, g represents the accelerometer and gyroscope respectively. The normalized accelerometer data D anorm Expressed as Finally, the normalized data from both sensors can be compiled into a single matrix D norm =[d anorm , d gnorm ] T , where D gnorm The treatment method is similar to D anorm This preprocessing step ensures that the data is consistently prepared during the subsequent training and authentication phases of the AuthEncoder.

[0076] 2. Data enhancement based on VAE

[0077] Variational autoencoders (VAEs) are powerful tools that can learn low-dimensional latent spaces in an unsupervised manner, making them ideal for representing high-dimensional, complex data. VAEs have been widely used in various fields, including image generation, image reconstruction and restoration, anomaly detection, and image data compression.

[0078] A typical VAE architecture consists of two main components: an encoder and a decoder. The encoder maps the input data into a distribution in a latent space and outputs a vector of means and standard deviations. The decoder samples from this latent space to reconstruct the original data space. While the reconstructed data is very similar to the original data, there are often subtle differences.

[0079] Convolutional neural networks (CNNs) are used in the encoder because they can effectively process spatially structured data, such as local features read from mobile sensors. In the decoder, a fully connected network is used to emphasize global information, thereby ensuring robust feature integration. Considering the dataset is relatively simple compared to typical image data, the present invention designs a simplified VAE architecture. This design enables efficient training and faster model convergence, making it well-suited for the present invention's continuous identity verification system.

[0080] like Figure 2 As shown, the VAE architecture of the present invention consists of three components: encoder, latent space and decoder.

[0081] Encoder: Encoder such as Figure 2 As shown in Table 1, the network consists of two branches. The first branch encodes features using a series of convolutional networks, consisting of four consecutive convolutional blocks. Each block consists of a convolutional (Conv) layer, followed by batch normalization (BN) and a leaky rectified linear unit (LeakyReLU) activation function. The convolution kernel sizes of the four blocks are set to 32, 64, 128, and 1, respectively. After processing through these convolutional blocks, the data is reshaped and concatenated with the output of the second branch. In the second branch, the sensor data is first reshaped and then processed through a fully connected network. Convolutional neural networks (CNNs), known for their ability to efficiently process spatially structured data, excel at extracting spatial features from sensor data and identifying sequential patterns and local trends. In contrast, fully connected networks excel at identifying global or sequential features and capturing overall trends in sensor data. After processing through both branches, the outputs are concatenated to form a richer and more comprehensive feature representation. The process concludes with two fully connected layers to optimize the dimensionality of the encoded sensor data.

[0082] Table 1

[0083]

[0084] Latent space: The latent space consists of a mean vector and a standard deviation vector, such as Figure 2 As shown. Latent variables z It is obtained through the reparameterization process and then passed to the decoder for reconstruction.

[0085] Decoder: Since the dataset of the present invention contains relatively low-complexity sensor readings, an overly complex decoder design is avoided. The present invention uses a fully connected network for decoding. Figure 2 As shown in Table 2, the decoder begins with a fully connected layer, followed by a ReLU activation function. This is followed by another fully connected layer, and finally ends with a Sigmoid activation function. Finally, the data is reshaped to match the format of the original sensor data, thus generating augmented data.

[0086] Table 2

[0087]

[0088] 3. Data reconstruction based on MulMemAE

[0089] This paper designs a multi-level memory-enhanced autoencoder (MulMemAE) to improve the effectiveness of user authentication systems. Figure 3As shown in Figure 1, the MulMemAE architecture consists of four components: an LSTM temporal feature extractor, an encoder, a multi-level feature memory module, and a decoder. Unlike traditional autoencoders and memory-augmented autoencoders, the MulMemAE architecture of the present invention integrates multi-level memory modules. A single memory unit may not effectively capture all normal patterns and may inadvertently reconstruct abnormal inputs. Multi-level memory modules are a strategic enhancement designed to alleviate these problems. However, excessive addition of memory units may lead to over-filtering, thereby reducing the model's ability to retain the most representative normal patterns.

[0090] The present invention addresses this issue by introducing skip connections between encoders and decoders at different layers. These skip connections allow direct access to encoded information at different levels, ensuring that important high-level encoded features are preserved. Furthermore, relying solely on convolutional architectures is insufficient to effectively extract temporal features from time series data. To address this issue, the present invention incorporates an LSTM module before the encoder to better capture temporal dynamics.

[0091] LSTM Time Series Information Extractor: This module consists of a single-layer LSTM network specifically designed to capture time series information from the input sensor data. The output of the LSTM extractor is then directly fed into the encoder for further processing. Before the sensor data is passed to the LSTM extractor, it undergoes transformations, including splitting and concatenation, compression, and transposition. After processing by the LSTM extractor, the data is transformed using the inverse operations, including transposition and expansion, and splitting and concatenation.

[0092] Encoder and Decoder: The encoder is responsible for converting the temporal features from the LSTM extractor into a latent representation. These encoded features serve as queries for retrieving relevant items from memory. The encoder consists of four layers, beginning with two consecutive Convolution + Batch Normalization + ReLU (Conv+BN+ReLU) blocks. This is followed by three downsampling blocks, each consisting of a stride-2 convolution layer and two Convolution + Batch Normalization + ReLU blocks. Each downsampling block performs downsampling, generating encoded features of varying resolutions at each step. The decoder's structure is symmetrical to the encoder, utilizing a weighted sum of various memory items. The decoder consists of three upsampling blocks and a convolutional layer. Each upsampling block accepts input from two sources: 1) the corresponding skip connection from the encoder and 2) upsampled features from a lower layer. These inputs are concatenated and processed through a deconvolution layer (ConvTranspose2d), followed by two Convolution + Batch Normalization + ReLU blocks and a memory module. The final convolutional layer reshapes the reconstructed data back to its original form, completing the reconstruction process.

[0093] Multi-level feature memory: Multi-level feature memory is used to store the encoded feature representations from different levels of the encoder. Each memory module consists of a matrix containing N real-valued vectors, each of dimension C, represented as Each row m of the matrix i (where i = 1, 2, 3, ..., N) represents a single memory item, and the value of N is set to 100. During the initial training phase, each memory item is initialized with a random value. As training progresses, the training data is first processed by the LSTM temporal information extractor and then passed to the encoder to generate the encoded features ef. The similarity between ef and each memory item is calculated using formula (1). Next, the weight of each item is determined using formula (2) using the Softmax function:

[0094]

[0095] wi=Soft max(Similarity(ef, m i )) (2)

[0096] Dense weights w may inadvertently lead to the reconstruction of abnormal inputs by combining multiple memory items. To address this problem, the present invention uses sparse addressing in formula (3) to refine and adjust the weight distribution:

[0097]

[0098] Among them, ∈ represents a very small positive number, As the threshold. After reduction, the present invention uses formula (4) to Perform normalization:

[0099]

[0100] The stored latent feature lf retrieved from memory can be expressed as formula (5):

[0101]

[0102] Sparse addressing forces the model to utilize a combination of fewer but more relevant memory items during data reconstruction. This strategy effectively addresses the challenge of accurately reconstructing anomalies and limits the use of multiple small components that may otherwise lead to the reconstruction of anomalous data. In each layer, the output of the memory-based representation is decoded by its respective decoder. These decoded outputs are fused through an upsampling operation to generate a reconstructed sensor sequence. The reconstruction error is calculated by comparing the reconstructed sequence with the original input. Typically, the reconstruction error of legitimate users is lower than that of imposters. The present invention establishes a predefined threshold by finding the optimal balance between the false positive rate (FAR) and the false negative rate (FRR). Users with reconstruction errors below this threshold are classified as legitimate users, while users with reconstruction errors exceeding this threshold are identified as imposters.

[0103] 4. User Authentication

[0104] In the authentication module, AuthEncoder determines the user's identity based on the reconstruction error, which quantifies the difference between the input sensor data sequence and the data sequence reconstructed by MulMemAE. The authentication threshold of AuthEncoder is set to achieve the best balance between the false positive rate (FAR) and the false negative rate (FRR). If the reconstruction error is lower than the threshold, the user will be authenticated as a legitimate user; otherwise, the user will be classified as an imposter. The reconstruction error is calculated using the mean square error (MSE), as shown in formula (6):

[0105]

[0106] Where X represents the input sensor data sequence, represents the data sequence reconstructed by MulMemAE, and i, j, and k represent the data dimensions.

[0107] In our experiments, AuthEncoder uses sensor data captured within a 2-second window for authentication, which serves as the basic segment for verification. We calculate the mean square error (MSE) of the data within this window and compare it with a predefined threshold to determine the user's identity.

[0108] This paper evaluates the effectiveness of VAE-based data augmentation by comparing the performance of AuthEncoder with and without VAE-based data augmentation at different data scales. In the experiment, one of 70 users was randomly selected as the legitimate user, while imposter data was equally sampled from the remaining 69 users. The model was then trained and tested, and the final experimental results were the average of the 70 users. Figure 4The results show that the model of the present invention has the same error rate (EER), accuracy and F1 score when using and not using VAE based data enhancement in the data size range from 50 to 500. Figure 4 As shown in Figure 2, the VAE model based on data augmentation consistently outperforms the model without augmentation. In general, increasing the data size will improve performance, although in some cases, larger datasets may cause a slight decrease due to the random nature of sampling. The improvement brought by data augmentation is most significant when the data size is 100. Compared with the model without augmentation, the VAE model based on augmentation has a better EER ( Figure 4 (a)), accuracy ( Figure 4 (b)) and F1 score ( Figure 4 (c)) all showed better performance, confirming the effectiveness of the VAE-based data augmentation method proposed in this invention. In addition, Table 3 lists the EER, false positive rate (FAR), false negative rate (FRR), accuracy, and F1 score of the AuthEncoder with and without enhancement at different data scales. In particular, when the data scale is 100, the VAE-based model's EER is reduced by 0.99%, FAR is reduced by 0.92%, and FRR is reduced by 1.05%, while the accuracy is improved by 1.01% and the F1 score is improved by 0.87%. In addition, to further verify the effectiveness of VAE-based data augmentation, this invention compares it with representative data augmentation methods, including deep learning-based methods such as CTGAN and CWGAN, and geometric transformation-based methods such as cropping and dithering. For deep learning-based enhancement methods, this invention first trains the model using real user data to ensure that it can generate high-quality enhanced data. In contrast, geometric enhancement-based methods directly apply basic geometric transformations to the data to create enhanced samples. This invention applies these enhancement methods to AuthEncoder under the same experimental conditions to ensure a fair comparison. The comparison results are as follows Figure 5 and as shown in Table 3.

[0109] Table 3

[0110]

[0111]

[0112] exist Figure 5 The EER, accuracy and F1 score of representative data augmentation methods are shown in Figure 2. Figure 5 As shown, VAE-based enhancement models generally achieve lower EER ( Figure 5 (a)), higher accuracy ( Figure 5 (b)) and higher F1 score ( Figure 5(c)). In contrast, the geometric transformation-based methods (cropping and dithering) have limited adaptability to the sensor data of the present invention, with cropping leading to only a slight performance improvement and dithering slightly degrading the performance. As shown in Table 3, CWGAN and cropping achieve the best performance at a data scale of 500. CWGAN achieves an EER of 2.80%, a FAR of 2.80%, an FRR of 2.80%, an accuracy of 97.20%, and an F1 score of 97.93%. Cropping achieves an EER of 3.01%, a FAR of 2.96%, an FRR of 3.06%, an accuracy of 96.97%, and an F1 score of 97.76%. For dithering, the best results occur at a data scale of 200, with an EER of 3.65%, a FAR of 3.62%, an FRR of 3.68%, an accuracy of 96.34%, and an F1 score of 97.29%. CTGAN achieved the best performance at a data scale of 300, with an EER of 2.87%, a FAR of 2.83%, an FRR of 2.91%, an accuracy of 97.12%, and an F1 score of 97.87%. In contrast, the VAE-based model performed best at a data scale of 500, with an EER of 2.66%, a FAR of 2.66%, an FRR of 2.67%, an accuracy of 97.33%, and an F1 score of 98.03%. These results further validate the superiority of the VAE-based data augmentation method of our invention.

[0113] To evaluate the efficiency of MulMemAE, we compared it with representative reconstruction models, including a standard autoencoder (AE) and a single memory-augmented autoencoder (SinMemAE). In the AE, we retained only the encoder and decoder components of MulMemAE; in SinMemAE, we retained the encoder, decoder, and the deepest memory module in MulMemAE. These models process input samples, compute reconstruction errors, and perform authentication by comparing the errors to a predefined threshold.

[0114] In our experiments, one user was randomly selected from 70 as the legitimate user, and data for nn users were sampled from the remaining 69 participants as unseen users. The legitimate user data was taken from the test set, while the imposter data was uniformly sampled from unseen users at a 2:1 ratio to validate the performance of the reconstruction models. For SinMemAE and AE, we used the same experimental setup as MulMemAE, with the only difference being the replacement of the model.

[0115] The present invention compares the performance of MulMemAE, AE and SinMemAE in terms of EER, accuracy and F1 score by changing the number of unseen users. Figure 6 shown. Figure 6It shows that MulMemAE consistently outperforms AE and SinMemAE in all metrics, while SinMemAE outperforms AE. As the number of unseen users increases, the EER of all models gradually increases, while the accuracy and F1 score decrease.

[0116] Furthermore, Table 4 shows the EER, FAR, FRR, accuracy, and F1 score results for MulMemAE, AE, and SinMemAE under different numbers of unseen users. As can be seen from the table, MulMemAE consistently outperforms both AE and SinMemAE. Specifically, in terms of EER, MulMemAE achieves the best performance of 1.14% with 10 unseen users and 2.30% with 60 unseen users. Correspondingly, SinMemAE achieves EERs of 2.08% and 3.48%, respectively, while AE achieves significantly higher EERs of 11.99% and 13.65%, respectively. For 10 unseen users, MulMemAE's EER is 0.94% lower than SinMemAE and 10.85% lower than AE; for 60 unseen users, the improvements are 1.18% and 11.35%, respectively.

[0117] Table 4

[0118]

[0119] In terms of accuracy, MulMemAE achieves a maximum of 98.84% with 10 unseen users and a minimum of 97.69% with 60 unseen users. In comparison, SinMemAE achieves accuracies of 97.91% and 96.51%, while AE lags behind with 88.01% and 86.35%, respectively. MulMemAE improves the accuracy of SinMemAE and AE by 0.93% and 10.83% (with 10 unseen users), and by 1.18% and 11.34% (with 60 unseen users).

[0120] In terms of F1 score, MulMemAE achieved the highest score of 99.12% when there were 10 unseen users, and the lowest score of 98.25% when there were 60 unseen users. In comparison, SinMemAE achieved scores of 98.41% and 97.34%, while AE achieved scores of 90.61% and 89.27%. In these two cases, MulMemAE improved over SinMemAE by 0.71% and 0.91%, respectively, and over AE by 8.51% and 8.98%. These results demonstrate that MulMemAE consistently outperforms both AE and SinMemAE, proving it to be the most effective model in the AuthEncoder continuous authentication system.

[0121] In the above embodiments, references to "this embodiment" in the specification indicate that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in at least some embodiments, but not necessarily all embodiments. Multiple occurrences of "this embodiment" do not necessarily refer to the same embodiment.

[0122] In the above embodiments, although the invention has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to those skilled in the art based on the foregoing description. For example, other memory structures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed. The embodiments of the present invention are intended to encompass all such alternatives, modifications, and variations that fall within the broad scope of the appended claims.

[0123] This embodiment further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, any one of the methods in this embodiment is implemented.

[0124] This embodiment also provides an electronic terminal, including: a processor and a memory;

[0125] The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the terminal executes any one of the methods in this embodiment.

[0126] Regarding the computer-readable storage medium in this embodiment, those skilled in the art will appreciate that all or part of the steps in the aforementioned method embodiments can be implemented using hardware associated with the computer program. The aforementioned computer program can be stored in a computer-readable storage medium. When executed, the program performs the steps in the aforementioned method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0127] The electronic terminal provided in this embodiment includes a processor, a memory, a transceiver and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication with each other. The memory is used to store computer programs, the communication interface is used for communication, and the processor and the transceiver are used to run computer programs so that the electronic terminal executes the various steps of the above method.

[0128] In this embodiment, the memory may include a random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage.

[0129] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0130] The present invention can be used in a wide variety of general-purpose or special-purpose computing system environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments that include any of the above.

[0131] The present invention may be described in the general context of computer-executable instructions, such as program modules, executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media, including storage devices.

[0132] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions, which should all be included in the scope of the claims of the present invention.

Claims

1. A continuous identity authentication method based on a multi-level memory-enhanced autoencoder, characterized by: The following steps are involved: S1: Use motion sensors on mobile devices to collect user behavior data; S2: Normalize the collected user behavior data; S3: Build a data augmentation model based on variational autoencoder (VAE) and train it using the data from step S2. S4: Build a data reconstruction model based on the multi-level memory-enhanced autoencoder MulMemAE, and use the data in step S2 and the VAE-enhanced data for training; S5: Input the real-time collected and normalized user behavior data into the trained data reconstruction model to reconstruct the user behavior data; S6: Compare the reconstructed user behavior data with the input user behavior data and calculate the reconstruction error; S7: Compare the reconstruction error with a preset threshold. If the error is lower than the preset threshold, the user is a legitimate user; otherwise, the user is an imposter and is required to log in again for identity verification. The data enhancement model based on variational autoencoder (VAE) includes an encoder, a latent space and a decoder; The encoder is divided into two branches. The first branch encodes features using four consecutive convolutional blocks, which are then concatenated with the output of the second branch. Each convolutional block contains a convolutional layer, a batch normalization, and a leaky rectified linear unit (LeakyReLU) activation function. The second branch first reshapes the input data and then processes it through a fully connected network. After concatenating the output of the second branch with the output of the first branch, two fully connected layers are used to optimize the encoding dimension. The latent space is composed of a mean vector and a standard deviation vector, and the latent variables are obtained through the reparameterization process. , and then passed to the decoder for reconstruction; The decoder includes a fully connected layer, a ReLU activation function, a fully connected layer and a Sigmoid activation function connected in sequence, and finally outputs enhanced data; The data reconstruction model includes an LSTM time series feature extractor, an encoder, a multi-level feature memory module and a decoder; The LSTM time series feature extractor consists of a single-layer LSTM network. The input data is first transformed, then the LSTM network is used to capture the time series information, and finally the data is transformed through the inverse operation. The encoder is used to convert the temporal features from the LSTM temporal feature extractor into a latent representation; the encoder consists of four layers, starting with two consecutive convolution + batch normalization + ReLU blocks; followed by three downsampling blocks, each of which includes a convolution layer with a stride of 2 and two convolution + batch normalization + ReLU blocks; each downsampling block performs a downsampling operation, generating encoded features of different resolutions in each step; The decoder has a symmetrical structure to the encoder, consisting of three upsampling blocks and a convolutional layer. Each upsampling block accepts input from two sources: 1) the corresponding skip connection from the encoder, and 2) the upsampled features from the lower layer. These inputs are concatenated and processed through a deconvolution layer, followed by two convolution + batch normalization + ReLU blocks and a memory module; the final convolution layer reshapes the reconstructed data back to its original form, completing the reconstruction process; The multi-level feature memory module is used to store the encoded feature representations from different levels of the encoder; Each level of feature memory module consists of a A matrix of real-valued vectors, each of dimension , expressed as ; Each row of the matrix Represents a single memory item; In the initial training phase, each memory item is initialized with a random value; the training data is first processed by the LSTM temporal information extractor and then passed to the encoder to generate the encoded features. ; Calculated by the following formula Similarity with each memory item: The Softmax function is then used to determine the weight of each item using the following formula: Use sparse addressing to refine and adjust weight distribution: in, represents a very small positive number, as a threshold; After reduction, use the following formula to Perform normalization: Stored latent features retrieved from memory , expressed as: 。 2. The method for continuous identity authentication based on a multi-level memory-enhanced autoencoder according to claim 1, characterized in that: The motion sensor in step S1 includes an accelerometer and a gyroscope. Internal Collection motion sensor samples, each consisting of a six-dimensional vector Composition, of which 、 and Represents the three axes of each motion sensor, and Represents the accelerometer and gyroscope respectively.

3. The method for continuous identity authentication based on a multi-level memory-enhanced autoencoder according to claim 1, characterized in that: In step S2, each set of sensor data is represented as a matrix; for the accelerometer, the matrix The structure is: in ; The data from the accelerometer and gyroscope are represented as ; Each axis of each sensor data is normalized to (0,1] by the following formula: in represents the three axes of each sensor, Represents each sample, represent the accelerometer and gyroscope respectively; Normalized accelerometer data Expressed as ; Normalized data from the accelerometer and gyroscope are compiled into a single matrix .

4. The method for continuous identity authentication based on a multi-level memory-enhanced autoencoder according to claim 1, characterized in that: The reconstruction error is calculated using the mean square error (MSE), as shown in the following formula: in represents the input sensor data sequence, represents the data sequence reconstructed by MulMemAE, 、 and Indicates the data dimension.

5. The method for continuous identity authentication based on a multi-level memory-enhanced autoencoder according to claim 1, characterized in that: The preset threshold is set to achieve an optimal balance between the false positive rate FAR and the false negative rate FRR.

Images