Enhanced self-encoder continuous identity authentication method based on multi-level memory

Abstract

The invention relates to a continuous identity authentication method based on a multi-level memory enhanced auto-encoder, which belongs to the technical field of continuous authentication and comprises the following steps: S1, collecting user behavior data; s2, performing normalization processing on the user behavior data; s3, constructing a VAE-based data enhancement model, and performing training by using normalized data; s4, constructing a data reconstruction model based on MulMemAE, and performing training by using normalized data and VAE enhanced data; s5, inputting the user behavior data collected and normalized in real time into the trained data reconstruction model to reconstruct the user behavior data; s6, comparing the reconstructed user behavior data with the input user behavior data, and calculating a reconstruction error; s7, the reconstruction error is compared with a preset threshold value, and if the error is lower than the preset threshold value, the user is a legal user; on the contrary, if the user is a famous person, the user is required to log in again

Description

Technical Field

[0001] The invention belongs to the technical field of continuous authentication and relates to a continuous identity authentication method based on a multi-level memory enhanced autoencoder. Background Art

[0002] In today's information age, mobile devices such as smartphones, smartwatches and tablets have become an integral part of daily life, profoundly changing the way we communicate, access information, work and entertain ourselves. These devices have significantly improved the convenience and connectivity of modern life. Smartphones enable online banking, allowing us to easily manage our finances, transfer money and pay bills; they also facilitate email communication, giving us instant access to personal and professional communications, and support mobile payments for fast and secure transactions. Smartwatches focus on health monitoring, helping users maintain and improve their health by tracking metrics such as heart rate, steps and sleep patterns. Tablets are particularly useful in remote work, supporting productivity applications, video conferencing and collaboration tools, allowing users to work efficiently anytime, anywhere.

[0003] Although mobile devices have brought significant convenience to our daily lives, they have also raised multiple privacy issues. Exposure of information stored in mobile devices can lead to significant losses for users, highlighting the need for sound security mechanisms. Traditional authentication methods include knowledge-based and physiological-based methods. Knowledge-based methods, such as personal identification numbers (PINs), passwords, tokens, and graphic passwords, rely on users' memory but are vulnerable to various attacks, including password guessing, dictionary attacks, password leakage, and man-in-the-middle attacks. While physiological-based mechanisms, such as fingerprint, facial recognition, iris, and palm vein recognition, generally provide higher security, they can also be subject to forgery and shadow attacks. These two traditional authentication methods only provide security at login time and are ineffective once an attacker gains access during device use. This vulnerability further highlights the urgent need to develop more continuous and powerful protection mechanisms to safeguard personal information on mobile devices.

[0004] Continuous authentication is an implicit approach that enhances security by continuously monitoring and verifying the identity of users as they interact with devices, thereby preventing unauthorized access and attacks. The approach leverages user behavioral characteristics, such as gait, chest movement, and touch gestures, which are involuntary and usually exhibit consistent patterns. These behaviors occur naturally and do not require conscious effort from the user, making continuous authentication both feasible and user-friendly. Continuous authentication systems typically include two main phases: enrollment and verification. In the enrollment phase, mobile devices capture behavioral data of authorized users through built-in sensors and use these data to train models to recognize patterns of individual users. In the verification phase, the system periodically collects sensor data to determine whether the current user is a legitimate user of the device. If the user is confirmed to be legitimate, access continues uninterrupted; otherwise, the system locks the device and requires re-login, thereby ensuring the security of user information on the device.

[0005] Most studies on continuous authentication use a feature extractor-classifier architecture, where the feature extractor projects sensor data into a higher-level feature representation, and then the classifier uses these features and corresponding labels to distinguish between legitimate and unauthorized users. At the same time, reconstruction-based methods are also emerging as a new type of continuous authentication method. These methods include a training phase, where the model encodes and decodes the original sensor data, generates reconstructed data, and calculates the reconstruction error to continuously optimize the model. If the reconstruction error is lower than a preset threshold in the verification phase, the user is considered legitimate; otherwise, the user is considered unauthorized.

[0006] On the one hand, the feature extractor-classifier paradigm requires the training of feature extractors and classifiers separately, which increases the complexity of the overall training process, and insufficient training of any part will affect the verification effect. On the other hand, the reconstruction-based structure is still under development, and it is difficult to effectively distinguish normal and abnormal samples during the reconstruction process. Both feature extractor-classifier and reconstruction-based methods require a large amount of user data to ensure the training effect. To alleviate the problem of data scarcity, researchers have adopted techniques such as permutation, scaling, flipping, sampling, jittering, and cropping, similar to image enhancement methods. However, these techniques may not be fully adapted to sensor data and often lead to homogenized training samples. Therefore, some researchers have turned to using deep learning for data enhancement, mainly using generative adversarial networks (GANs). GANs train generator and discriminator models simultaneously, which can generate more diverse data, but can also lead to unstable training processes, such as mode collapse or gradient vanishing. In addition, GANs usually require a long training time, which may affect the real-time performance of user identity authentication. Summary of the invention

[0007] In view of this, the object of the present invention is to provide a continuous identity authentication method based on a multi-level memory enhanced autoencoder.

[0008] In order to achieve the above object, the present invention provides the following technical solutions:

[0009] A continuous identity authentication method based on a multi-level memory enhanced autoencoder comprises the following steps:

[0010] S1: Collect user behavior data using motion sensors on mobile devices;

[0011] S2: Normalize the collected user behavior data;

[0012] S3: Build a data augmentation model based on variational autoencoder VAE and train it using the data in step S2;

[0013] S4: Build a data reconstruction model based on the multi-level memory enhanced autoencoder MulMemAE, and use the data in step S2 and the VAE enhanced data for training;

[0014] S5: inputting the user behavior data collected and normalized in real time into the trained data reconstruction model to reconstruct the user behavior data;

[0015] S6: Compare the reconstructed user behavior data with the input user behavior data and calculate the reconstruction error;

[0016] S7: Compare the reconstruction error with a preset threshold. If the error is lower than the preset threshold, the user is a legitimate user; otherwise, the user is an impostor and is required to log in again for identity authentication.

[0017] 进一步，步骤S1中所述动作传感器包括加速度计和陀螺仪，在时间段 t 内收集 n 个动作 Sensor samples, where n = t × f, each sample consists of a six-dimensional vector where x, y, and z represent the three axes of each motion sensor, and a and g represent the accelerometer and gyroscope, respectively.

[0018] Further, in step S2, each set of sensor data is represented as a d×n matrix; for the accelerometer, the matrix D a The structure is:

[0019]

[0020] Where d = 3 and n = t × f;

[0021] The data of the accelerometer and gyroscope are expressed as D = [D a , D g ];

[0022] Each axis of each sensor data is normalized to (0, 1] by the following formula:

[0023]

[0024] Where i = x, y, z represents the three axes of each sensor, j = 1, 2, ..., n represents each sample, and s = a, g represents the accelerometer and gyroscope respectively;

[0025] Normalized accelerometer data D anorm Expressed as

[0026] The normalized data from the accelerometer and gyroscope are compiled into a single matrix D norm =[d anorm , d gnorm ] T .

[0027] Furthermore, the data enhancement model based on variational autoencoder VAE includes an encoder, a latent space and a decoder;

[0028] The encoder is divided into two branches. The first branch encodes features using four consecutive convolution blocks and then connects them with the output of the second branch. Each convolution block contains a convolution layer, a batch normalization and a leaky rectified linear unit (LeakyReLU) activation function. The second branch first reshapes the input data and then processes it through a fully connected network. After connecting the output of the second branch with the output of the first branch, two fully connected layers are used to optimize the encoding dimension.

[0029] The latent space consists of a mean vector and a standard deviation vector, and the latent variables are obtained through the reparameterization process. z , and then passed to the decoder for reconstruction;

[0030] The decoder includes a fully connected layer, a ReLU activation function, a fully connected layer and a Sigmoid activation function connected in sequence, and finally outputs enhanced data.

[0031] Furthermore, the data reconstruction model includes an LSTM time series feature extractor, an encoder, a multi-level feature memory module and a decoder;

[0032] The LSTM time series feature extractor consists of a single-layer LSTM network; the input data is first transformed, then the time series information is captured using the LSTM network, and finally converted through an inverse operation;

[0033] The encoder is used to convert the temporal features from the LSTM temporal feature extractor into a latent representation; the encoder consists of four layers, first two consecutive convolution + batch normalization + ReLU (Conv+BN+ReLU) blocks. Next are three downsampling blocks, each of which includes a convolution layer with a stride of 2 and two convolution + batch normalization + ReLU blocks; each downsampling block performs a downsampling operation, generating encoding features of different resolutions in each step;

[0034] The structure of the decoder is symmetrical to that of the encoder, and includes three upsampling blocks (Up blocks) and a convolutional layer; each upsampling block accepts input from two sources: 1) the corresponding jump connection from the encoder, and 2) the upsampled features from the lower layer; after these inputs are connected, they are processed through a deconvolution layer (ConvTranspose2d), followed by two convolution + batch normalization + ReLU blocks and a memory module; the final convolutional layer resizes the reconstructed data to its original form to complete the reconstruction process;

[0035] The multi-level feature memory module is used to store the encoded feature representations from different levels of the encoder; each level of the feature memory module consists of a matrix containing N real-valued vectors, each of which has a dimension of C and is expressed as Each row m of the matrix i represents a single memory item; in the initial training phase, each memory item is initialized with a random value; the training data is first processed by the LSTM temporal information extractor and then passed to the encoder to generate the encoding feature ef; the similarity between ef and each memory item is calculated by the following formula:

[0036]

[0037] The Softmax function is then used to determine the weight of each item using the following formula:

[0038] w i =Softmax(Similarity(ef,m i ))

[0039] Use sparse addressing to refine and adjust weight distribution:

[0040]

[0041] Among them, ∈ represents a very small positive number, as a threshold;

[0042] After reduction, use the following formula to Normalize:

[0043]

[0044] The stored latent feature lf retrieved from memory is expressed as:

[0045]

[0046] Furthermore, the reconstruction error is calculated using mean square error (MSE), as shown in the following formula:

[0047]

[0048] Where X represents the input sensor data sequence, represents the data sequence reconstructed by MulMemAE, and i, j and k represent the data dimensions.

[0049] Furthermore, the preset threshold is set to achieve an optimal balance between the false positive rate (FAR) and the false negative rate (FRR).

[0050] The beneficial effects of the present invention are:

[0051] 1) This paper proposes a data augmentation method based on VAE, which integrates convolutional neural network (CNN) and fully connected network. This method enhances the stability of training and reduces the time required to generate augmented data, making the training process more efficient.

[0052] 2) The present invention adopts an identity authentication method based on single-stage reconstruction. This method combines feature extraction and classification into one step, which not only simplifies the system but also improves the overall efficiency of the model.

[0053] 3) The present invention uses MulMemAE to capture spatial and temporal features through multi-level feature memory. The skip connection between the encoder and decoder ensures that key high-level features are preserved, thereby improving the model's ability to distinguish legitimate data from imposter data.

[0054] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0056] Figure 1This is the AuthEncoder architecture diagram;

[0057] Figure 2 This is the architecture diagram of VAE;

[0058] Figure 3 This is the structural diagram of MulMemAE;

[0059] Figure 4 (a)-(c) are the EER, accuracy and F1Score graphs with and without VAE under different data amounts;

[0060] Figure 5 (a)-(c) are the EER, Accuracy and F1Sco under different data scales and different data enhancement methods. re picture;

[0061] Figure 6 (a)-(c) are the EER, accuracy and F1 Score of AuthEncoder using MulMemAE, AE and SinMemAE on different unseen users respectively; DETAILED DESCRIPTION

[0062] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and the features in the embodiments can be combined with each other without conflict.

[0063] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0064] In the following description, numerous details are discussed to provide a more thorough explanation of the embodiments of the present invention. However, it is obvious to those skilled in the art that the embodiments of the present invention can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present invention difficult to understand.

[0065] The present invention provides a method for continuous identity authentication based on a multi-level memory-enhanced autoencoder. The system constructed by the method is called a reconstruction-based continuous identity authentication system AuthEncoder, which uses a variational autoencoder (VAE) for training data enhancement and designs a multi-level memory-enhanced autoencoder (MulMemAE) for data reconstruction. AuthEncoder consists of four modules: data collection and preprocessing, VAE-based enhancement, data reconstruction and identity authentication. The operation process of AuthEncoder includes a registration phase and a verification phase.

[0066] During the registration phase, AuthEncoder uses the accelerometer and gyroscope embedded sensors on mobile devices to capture user behavior patterns. The collected sensor data is normalized and used to train VAE for data augmentation and integrated with convolutional neural networks (CNNs) and fully connected networks. Subsequently, the original data and VAE-enhanced data are used to train MulMemAE for data reconstruction, which introduces skip connections between encoders and decoders at different layers.

[0067] During the verification phase, when the user interacts with the mobile device, sensor data will be collected and input into the trained MulMemAE to reconstruct the corresponding user data. The reconstruction error is calculated by comparing the reconstructed data with the current input. Based on the reconstruction error, AuthEncoder identifies the current user by comparing it with a preset threshold: if the error is lower than the threshold, the system identifies the user as a legitimate user; if the error exceeds the threshold, the user is considered an imposter and the system will ask the user to log in again.

[0068] AuthEncoder consists of two components: a device client and a remote server. The device client is responsible for the data collection and authentication process, while the remote server is responsible for the training of the VAE and MulMemAE models. This division of labor ensures that the client can efficiently process data during the authentication process, while placing the resource-intensive model training tasks on the server. As a result, mobile devices are able to avoid complex computational burdens, thereby optimizing performance and improving the overall user experience.

[0069] The AuthEncoder architecture consists of four modules: data collection and preprocessing, data enhancement based on VAE, data reconstruction based on MulMemAE, and user authentication. Figure 1As shown in the figure. In the data collection and preprocessing module, the accelerometer and gyroscope embedded in the mobile device collect user behavior data. In the registration phase, the collected data is normalized and securely transmitted to the remote server. On the server, the VAE model is trained using the transmitted data to achieve data enhancement, thereby improving the robustness of the system. In the authentication phase, the data is processed locally without being transmitted over the network.

[0070] In the data reconstruction module, MulMemAE is trained using both original data and VAE-enhanced data. The training guides the parameter update in MulMemAE by optimizing the reconstruction error and the entropy loss in the memory item weights. After training, the trained MulMemAE parameters are returned to the mobile device for user authentication. It should be noted that both enhanced and original data are used during MulMemAE training, while only original data is used during the testing phase.

[0071] In the user authentication module, once the user interacts with the device, the system collects and normalizes sensor data in real time. This data is processed by a trained MulMemAE, which includes a long short-term memory (LSTM) network to capture temporal information, an encoder, multi-level memory, and a decoder to reconstruct the input. The system then compares the reconstruction error between the current input and the reconstructed data. If the error is below a predefined threshold, the user will be considered legitimate; otherwise, the user will be classified as an imposter and will need to log in again for authentication.

[0072] 1. Data collection and preprocessing

[0073] Data is collected from embedded sensors in mobile devices, including accelerometers and gyroscopes, to capture user behavior patterns. Specifically, accelerometers measure users' coarse behavioral movements, while gyroscopes capture finer user movements. For the dataset, accelerometer and gyroscope readings were collected from 100 participants (53 males and 47 females) with a sampling rate of f = 100 Hz, and 100 minutes of data were collected for each participant. When a user interacts with the device, AuthEncoder collects n sensor data samples in time period t, where n = t×f. Each sample consists of a six-dimensional vector where x, y, and z represent the three axes of each sensor, and a and g represent the accelerometer and gyroscope, respectively.

[0074] In the data preprocessing module, n raw data samples are collected from the accelerometer and gyroscope within a given time period t. Each set of sensor data can be represented as a d×n matrix. For the accelerometer, the matrix D a The structure is

[0075] Where d = 3 and n = t × f. The data of the accelerometer and gyroscope can be expressed as D = [D a , D g ]. To prepare the data for training VAE and MulMemAE models, each axis of each sensor data is normalized to (0, 1] by the following formula: Where i = x, y, z represents the three axes of each sensor, j = 1, 2, ..., n represents each sample, and s = a, g represents the accelerometer and gyroscope respectively. Normalized accelerometer data D anorm Expressed as Finally, the normalized data from both sensors can be compiled into a single matrix D norm =[d anorm , d gnorm ] T , where D gnorm The treatment method is similar to D anorm This preprocessing step ensures that the data is consistently prepared during the subsequent training and authentication phases of AuthEncoder.

[0076] 2. Data enhancement based on VAE

[0077] Variational Autoencoders (VAEs) are powerful tools that can learn low-dimensional latent spaces in an unsupervised manner, making them ideal for representing high-dimensional complex data. VAEs have been widely used in many fields, including image generation, image reconstruction and restoration, anomaly detection, and image data compression.

[0078] A typical VAE architecture consists of two main components: an encoder and a decoder. The encoder is responsible for mapping the input data to a distribution in a latent space and outputs a mean and standard deviation vector. The decoder is able to sample from this latent space to reconstruct the original data space. Although the reconstructed data is very similar to the original data, there are usually some subtle differences.

[0079] Since convolutional neural networks (CNNs) can effectively process spatially structured data, such as local features read from mobile sensors, they are used in the encoder. In the decoder, a fully connected network is used to emphasize global information, thereby ensuring the robustness of feature integration. Considering that the dataset is relatively simple compared to typical image data, the present invention designs a simplified VAE architecture. This design can achieve efficient training and faster model convergence, making it very suitable for the continuous identity authentication system of the present invention.

[0080] like Figure 2 As shown, the VAE architecture of the present invention consists of three components: encoder, latent space and decoder.

[0081] Encoder: Encoder such as Figure 2 As shown in Table 1, it is divided into two branches. The first branch encodes features using a series of convolutional networks, including four consecutive convolutional blocks. Each block contains a convolutional (Conv) layer followed by batch normalization (BN) and leaky rectified linear unit (LeakyReLU) activation functions. The convolution kernel sizes of the four blocks are set to 32, 64, 128, and 1, respectively. After processing these convolutional blocks, the data is reshaped and concatenated with the output of the second branch. In the second branch, the sensor data is first reshaped and then processed through a fully connected network. Convolutional neural networks (CNNs) excel in extracting spatial features, identifying sequential patterns, and local trends from sensor data due to their ability to efficiently process spatially structured data. In contrast, fully connected networks are good at identifying global or sequential features and capturing overall trends in sensor data. After processing in both branches, the outputs are concatenated together to form a richer and more comprehensive feature representation. The process ends with two fully connected layers to optimize the dimensions of the encoded sensor data.

[0082] Table 1

[0083]

[0084] Latent space: The latent space consists of a mean vector and a standard deviation vector, such as Figure 2 As shown. Latent variables z It is obtained through the reparameterization process and then passed to the decoder for reconstruction.

[0085] Decoder: Since the dataset of the present invention contains relatively low-complexity sensor readings, an overly complex decoder design is avoided. The present invention uses a fully connected network for decoding. Figure 2 As shown in Table 2, the decoder starts with a fully connected layer, followed by a ReLU activation function. This is followed by another fully connected layer and ends with a Sigmoid activation function. Finally, the data is reshaped to match the format of the original sensor data to generate augmented data.

[0086] Table 2

[0087]

[0088] 3. Data reconstruction based on MulMemAE

[0089] This paper designs a multi-level memory-enhanced autoencoder (MulMemAE) to improve the effectiveness of user identity authentication systems. Figure 3As shown, the MulMemAE architecture includes four components: an LSTM time series feature extractor, an encoder, a multi-level feature memory module, and a decoder. Unlike traditional autoencoders and memory-enhanced autoencoders, the MulMemAE architecture of the present invention integrates multi-level memory modules. A single memory unit may not be able to effectively capture all normal patterns and may inadvertently reconstruct abnormal inputs. The multi-level memory module is a strategic enhancement designed to alleviate these problems. However, an excessive increase in memory units may lead to over-filtering, thereby reducing the model's ability to retain the most representative normal patterns.

[0090] The present invention solves this problem by introducing skip connections between encoders and decoders at different layers. These skip connections allow direct access to the encoded information at different levels, thus ensuring that important high-level encoded features are preserved. In addition, relying solely on convolutional structures is not enough to effectively extract the temporal features of time series data. To solve this problem, the present invention adds an LSTM module before the encoder to better capture the temporal dynamics.

[0091] LSTM Time Series Information Extractor: This module consists of a single-layer LSTM network that is specifically designed to capture time series information from the input sensor data. The output of the LSTM extractor is then directly input to the encoder for further processing. Before the sensor data is passed to the LSTM extractor, the data undergoes transformations, including splitting and concatenation, and compression and transposition operations. After being processed by the LSTM extractor, the data is transformed through inverse operations, including transposition and expansion, and splitting and concatenation.

[0092] Encoder and Decoder: The encoder is responsible for converting the temporal features from the LSTM extractor into latent representations. These encoded features serve as queries to retrieve relevant items from the memory. The encoder consists of four layers, starting with two consecutive Convolution + Batch Normalization + ReLU (Conv+BN+ReLU) blocks. This is followed by three Down blocks, each of which consists of a Convolution layer with a stride of 2 and two Convolution + Batch Normalization + ReLU blocks. Each Down block performs a downsampling operation, generating encoded features of different resolutions at each step. The structure of the decoder is symmetrical to that of the encoder, utilizing a weighted sum of various memory items. The decoder contains three Up blocks and a Convolution layer. Each Up block accepts inputs from two sources: 1) the corresponding skip connection from the encoder, and 2) the upsampled features from the lower layers. These inputs are concatenated and processed through a Deconvolution layer (ConvTranspose2d), followed by two Convolution + Batch Normalization + ReLU blocks and a Memory module. The final Convolution layer reshapes the reconstructed data back to its original form to complete the reconstruction process.

[0093] Multi-level feature memory: Multi-level feature memory is used to store the encoded feature representations from different levels of the encoder. Each memory module consists of a matrix containing N real-valued vectors, each of which has a dimension of C and is expressed as Each row m of the matrix i (where i = 1, 2, 3, ..., N) represents a single memory item, and the value of N is set to 100. In the initial training phase, each memory item is initialized with a random value. As training progresses, the training data is first processed by the LSTM temporal information extractor and then passed to the encoder to generate the encoded feature ef. The similarity between ef and each memory item is calculated using formula (1). Next, the weight of each item is determined using formula (2) using the Softmax function:

[0094]

[0095] wi=Soft max(Similarity(ef, m i )) (2)

[0096] Dense weights w may inadvertently lead to reconstruction of abnormal inputs by combining multiple memory items. To address this problem, the present invention uses sparse addressing in formula (3) to refine and adjust the weight distribution:

[0097]

[0098] Among them, ∈ represents a very small positive number, As a threshold. After reduction, the present invention uses formula (4) to Normalize:

[0099]

[0100] The stored latent feature lf retrieved from the memory can be expressed as formula (5):

[0101]

[0102] Sparse addressing forces the model to utilize a combination of fewer but more relevant memory items during data reconstruction. This strategy effectively addresses the challenge of accurately reconstructing anomalies and limits the use of multiple small components that may otherwise result in reconstructing anomalous data. In each layer, the outputs of the memory-based representations are decoded by their respective decoders. These decoded outputs are fused through an upsampling operation to generate a reconstructed sensor sequence. The reconstruction error is calculated by comparing the reconstructed sequence with the original input. Typically, the reconstruction error of legitimate users is lower than that of impostors. The present invention establishes a predefined threshold by finding the optimal balance between the false positive rate (FAR) and the false negative rate (FRR). Users with reconstruction errors below this threshold are classified as legitimate users, while users exceeding this threshold are identified as impostors.

[0103] 4. User Authentication

[0104] In the authentication module, AuthEncoder determines the user identity based on the reconstruction error, which quantifies the difference between the input sensor data sequence and the data sequence reconstructed by MulMemAE. The authentication threshold of AuthEncoder is set to achieve the best balance between the false positive rate (FAR) and the false negative rate (FRR). If the reconstruction error is lower than the threshold, the user will be authenticated as a legitimate user; otherwise, the user will be classified as an imposter. The reconstruction error is calculated using the mean square error (MSE), as shown in formula (6):

[0105]

[0106] Where X represents the input sensor data sequence, represents the data sequence reconstructed by MulMemAE, and i, j and k represent the data dimensions.

[0107] In the experiment of the present invention, AuthEncoder uses sensor data captured within a 2-second window for identity authentication, which serves as the basic segment for authentication. The present invention calculates the mean square error (MSE) of the data within the window and compares it with a predefined threshold to determine the user's identity.

[0108] The present invention evaluates the effectiveness of VAE-based data enhancement by comparing the performance of AuthEncoder with and without VAE-based data enhancement at different data scales. In the experiment, the present invention randomly selects one of the 70 users as the legitimate user, and equally extracts the impostor data from the remaining 69 users. Subsequently, the model is trained and tested, and the final experimental result adopts the average result of 70 users. Figure 4The equal error rate (EER), accuracy and F1 score of the model of the present invention with and without VAE based data enhancement are shown in the data scale range from 50 to 500. Figure 4 As shown in Figure 2, the VAE model based on data augmentation consistently outperforms the model without augmentation. In general, increasing the data size will improve performance, although in some cases, larger datasets may cause a slight decrease due to the random nature of sampling. The improvement brought by data augmentation is most significant when the data size is 100. Compared with the model without augmentation, the VAE model based on augmentation has an EER ( Figure 4 (a)), accuracy ( Figure 4 (b)) and F1 score ( Figure 4 (c)) all show better performance, confirming the effectiveness of the VAE-based data enhancement method proposed in the present invention. In addition, Table 3 lists the EER, false positive rate (FAR), false negative rate (FRR), accuracy and F1 score of AuthEncoder with and without enhancement under different data scales. In particular, when the data scale is 100, the VAE model-based EER is reduced by 0.99%, FAR is reduced by 0.92%, and FRR is reduced by 1.05%, while the accuracy is improved by 1.01% and the F1 score is improved by 0.87%. In addition, in order to further verify the effectiveness of VAE based on data enhancement, the present invention compares it with representative data enhancement methods, including deep learning-based methods such as CTGAN and CWGAN, and geometric transformation-based methods such as cropping and jittering. For deep learning-based enhancement methods, the present invention first trains the model using real user data to ensure that it can generate high-quality enhanced data. In contrast, the method based on geometric enhancement directly applies basic geometric transformations to the data to create enhanced samples. The present invention applies these enhancement methods to AuthEncoder under the same experimental conditions to ensure fair comparison. The comparison results are as follows Figure 5 And as shown in Table 3.

[0109] Table 3

[0110]

[0111]

[0112] exist Figure 5 The EER, accuracy, and F1 score of representative data augmentation methods are shown in Figure 2, where the data size ranges from 50 to 500. Figure 5 As shown, VAE-based enhancement models generally achieve lower EER ( Figure 5 (a)), higher accuracy ( Figure 5 (b)) and higher F1 score ( Figure 5(c)). In contrast, the geometric transformation based methods (cropping and jittering) have limited adaptability to the sensor data of the present invention, with cropping leading to only a slight performance improvement and jittering slightly reducing the performance. As shown in Table 3, CWGAN and cropping achieve the best performance at a data scale of 500. CWGAN achieves an EER of 2.80%, a FAR of 2.80%, an FRR of 2.80%, an accuracy of 97.20%, and an F1 score of 97.93%. Cropping achieves an EER of 3.01%, a FAR of 2.96%, an FRR of 3.06%, an accuracy of 96.97%, and an F1 score of 97.76%. For jittering, the best results occur at a data scale of 200, with an EER of 3.65%, a FAR of 3.62%, an FRR of 3.68%, an accuracy of 96.34%, and an F1 score of 97.29%. CTGAN achieves the best performance at a data scale of 300, with an EER of 2.87%, a FAR of 2.83%, a FRR of 2.91%, an accuracy of 97.12%, and an F1 score of 97.87%. In contrast, the VAE-based model performs best at a data scale of 500, with an EER of 2.66%, a FAR of 2.66%, a FRR of 2.67%, an accuracy of 97.33%, and an F1 score of 98.03%. These results further verify the superiority of the VAE-based data augmentation method of the present invention.

[0113] To evaluate the efficiency of MulMemAE, we compare it with representative reconstruction models, including standard autoencoders (AE) and single memory-enhanced autoencoders (SinMemAE). In AE, we only retain the encoder and decoder components of MulMemAE; while in SinMemAE, we retain the encoder, decoder, and the deepest memory module in MulMemAE. These models process input samples, calculate reconstruction errors, and complete identity verification by comparing the errors with predefined thresholds.

[0114] In the experiment of the present invention, one user is randomly selected from 70 as the legitimate user, and the data of nn users are sampled from the remaining 69 participants as unseen users. The legitimate user data is taken from the test set, while the impostor data is uniformly sampled from the unseen users at a ratio of 2:1 to verify the performance of these reconstruction models. For SinMemAE and AE, the present invention uses the same experimental settings as MulMemAE, the only difference is the replacement of the model.

[0115] The present invention compares the performance of MulMemAE, AE and SinMemAE in terms of EER, accuracy and F1 score by changing the number of unseen users. Figure 6 shown. Figure 6It shows that MulMemAE consistently outperforms AE and SinMemAE in all metrics, while SinMemAE outperforms AE. As the number of unseen users increases, the EER of all models gradually increases, while the accuracy and F1 score decrease.

[0116] In addition, Table 4 shows the results of EER, FAR, FRR, accuracy, and F1 score for MulMemAE, AE, and SinMemAE under different numbers of unseen users. As can be seen from the table, MulMemAE consistently outperforms AE and SinMemAE. Specifically, in terms of EER, MulMemAE achieves the best performance of 1.14% when there are 10 unseen users and 2.30% when there are 60 unseen users. Correspondingly, the EERs of SinMemAE are 2.08% and 3.48%, respectively, while the corresponding EERs of AE are significantly higher, 11.99% and 13.65%, respectively. For 10 unseen users, the EER of MulMemAE is 0.94% lower than that of SinMemAE and 10.85% lower than that of AE; while for 60 unseen users, the improvements are 1.18% and 11.35%, respectively.

[0117] Table 4

[0118]

[0119] In terms of accuracy, MulMemAE reaches a maximum of 98.84% at 10 unseen users and a minimum of 97.69% at 60 unseen users. In comparison, SinMemAE achieves accuracies of 97.91% and 96.51%, while AE lags behind at 88.01% and 86.35%, respectively. MulMemAE improves the accuracy of SinMemAE and AE by 0.93% and 10.83% (10 unseen users), and 1.18% and 11.34% (60 unseen users).

[0120] In terms of F1 score, MulMemAE achieves the highest score of 99.12% when there are 10 unseen users and the lowest score of 98.25% when there are 60 unseen users. In comparison, SinMemAE scores 98.41% and 97.34%, while AE scores 90.61% and 89.27%. MulMemAE improves over SinMemAE by 0.71% and 0.91% in these two cases, and over AE by 8.51% and 8.98%. These results show that MulMemAE consistently outperforms AE and SinMemAE, proving to be the most effective model in the AuthEncoder continuous authentication system.

[0121] In the above embodiments, the description's reference to "this embodiment" indicates that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in at least some embodiments, but not necessarily all embodiments. Multiple occurrences of "this embodiment" do not necessarily all refer to the same embodiment.

[0122] In the above-described embodiments, although the invention has been described in conjunction with specific embodiments of the invention, many substitutions, modifications, and variations of these embodiments will be apparent to those of ordinary skill in the art based on the foregoing description. For example, other storage structures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed. Embodiments of the invention are intended to encompass all such substitutions, modifications, and variations that fall within the broad scope of the appended claims.

[0123] This embodiment further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, any one of the methods in this embodiment is implemented.

[0124] This embodiment also provides an electronic terminal, including: a processor and a memory;

[0125] The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the terminal executes any one of the methods in this embodiment.

[0126] The computer-readable storage medium in this embodiment can be understood by ordinary technicians in this field: all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to the computer program. The aforementioned computer program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes.

[0127] The electronic terminal provided in this embodiment includes a processor, a memory, a transceiver and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication with each other. The memory is used to store computer programs, the communication interface is used to communicate, and the processor and the transceiver are used to run computer programs so that the electronic terminal executes each step of the above method.

[0128] In this embodiment, the memory may include a random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0129] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0130] The present invention can be used in many general or special computing system environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like.

[0131] The present invention may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0132] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A continuous identity authentication method based on a multi-level memory-enhanced autoencoder, characterized in that: The following steps are involved: S1: Collect user behavior data using motion sensors on mobile devices; S2: Normalize the collected user behavior data; S3: Build a data augmentation model based on variational autoencoder VAE and train it using the data in step S2; S4: Build a data reconstruction model based on the multi-level memory enhanced autoencoder MulMemAE, and use the data in step S2 and the VAE enhanced data for training; S5: inputting the user behavior data collected and normalized in real time into the trained data reconstruction model to reconstruct the user behavior data; S6: Compare the reconstructed user behavior data with the input user behavior data and calculate the reconstruction error; S7: Compare the reconstruction error with a preset threshold. If the error is lower than the preset threshold, the user is a legitimate user; otherwise, the user is an impostor and is required to log in again for identity authentication.

2. The method for continuous identity authentication based on multi-level memory enhanced autoencoder according to claim 1, characterized in that: The motion sensor in step S1 includes an accelerometer and a gyroscope. n motion sensor samples are collected in a time period t, where n = t × f, and each sample consists of a six-dimensional vector where x, y, and z represent the three axes of each motion sensor, and a and g represent the accelerometer and gyroscope, respectively.

3. The method for continuous identity authentication based on multi-level memory enhanced autoencoder according to claim 1, characterized in that: In step S2, each set of sensor data is represented as a d×n matrix; for the accelerometer, the matrix D a The structure is: Where d = 3 and n = t × f; The data of the accelerometer and gyroscope are expressed as D = [D a , D g ]; Each axis of each sensor data is normalized to (0,1] by the following formula: Where i = x, y, z represents the three axes of each sensor, j = 1, 2, ..., n represents each sample, and s = a, g represents the accelerometer and gyroscope respectively; Normalized accelerometer data D anorm Expressed as The normalized data from the accelerometer and gyroscope are compiled into a single matrix D norm =[d anorm , d gnorm ] T .

4. The method for continuous identity authentication based on multi-level memory enhanced autoencoder according to claim 1, characterized in that: The data enhancement model based on variational autoencoder VAE includes an encoder, a latent space and a decoder; The encoder is divided into two branches. The first branch encodes features using four consecutive convolution blocks and then connects them with the output of the second branch. Each convolution block contains a convolution layer, a batch normalization and a leaky rectified linear unit (LeakyReLU) activation function. The second branch first reshapes the input data and then processes it through a fully connected network. After connecting the output of the second branch with the output of the first branch, two fully connected layers are used to optimize the encoding dimension. The latent space is composed of a mean vector and a standard deviation vector, and the latent variable z is obtained through a reparameterization process and then passed to the decoder for reconstruction; The decoder includes a fully connected layer, a ReLU activation function, a fully connected layer and a Sigmoid activation function connected in sequence, and finally outputs enhanced data.

5. The method for continuous identity authentication based on multi-level memory enhanced autoencoder according to claim 1, characterized in that: The data reconstruction model includes an LSTM time series feature extractor, an encoder, a multi-level feature memory module and a decoder; The LSTM time series feature extractor consists of a single-layer LSTM network; the input data is first transformed, then the time series information is captured using the LSTM network, and finally converted through an inverse operation; The encoder is used to convert the temporal features from the LSTM temporal feature extractor into a latent representation; the encoder consists of four layers, first of which are two consecutive convolution + batch normalization + ReLU blocks; followed by three downsampling blocks, each of which includes a convolution layer with a stride of 2 and two convolution + batch normalization + ReLU blocks; each downsampling block performs a downsampling operation to generate encoding features of different resolutions in each step; The structure of the decoder is symmetrical to that of the encoder, consisting of three upsampling blocks and a convolutional layer; each upsampling block accepts input from two sources: 1) the corresponding skip connection from the encoder, 2) the upsampled features from the lower layer; These inputs are concatenated and processed through a deconvolution layer, followed by two convolution + batch normalization + ReLU blocks and a memory module; the final convolution layer reshapes the reconstructed data back to its original form, completing the reconstruction process; The multi-level feature memory module is used to store the encoded feature representations from different levels of the encoder; Each level of feature memory module consists of a matrix containing N real-valued vectors, each of which has a dimension of C and is expressed as Each row m of the matrix i Represents a single memory item; In the initial training phase, each memory item is initialized with a random value; the training data is first processed by the LSTM temporal information extractor and then passed to the encoder to generate the encoding feature ef; the similarity between ef and each memory item is calculated by the following formula: The Softmax function is then used to determine the weight of each item using the following formula: w i =Softmax(Similarity(ef,m i )) Use sparse addressing to refine and adjust weight distribution: Among them, ∈ represents a very small positive number, as a threshold; After reduction, use the following formula to Normalize: The stored latent feature lf retrieved from memory is expressed as:

6. The method for continuous identity authentication based on multi-level memory enhanced autoencoder according to claim 1, characterized in that: The reconstruction error is calculated using the mean square error MSE, as shown in the following formula: Where X represents the input sensor data sequence, represents the data sequence reconstructed by MulMemAE, and i, j and k represent the data dimensions.

7. The method for continuous identity authentication based on multi-level memory enhanced autoencoder according to claim 1, characterized in that: The preset threshold is set to achieve an optimal balance between the false positive rate FAR and the false negative rate FRR.

Images