Continuous identity authentication method based on multichannel PPG signals in uncontrolled environment

Abstract

The invention provides a continuous identity authentication method based on multichannel PPG signals in an uncontrolled environment. Firstly, a smart watch is used for collecting wrist double-channel green light, red light and infrared light PPG signals and motion sensor data, and original information is obtained. Aiming at the problem that an uncontrolled environment signal is easily interfered, noise reduction and optimization are carried out on the signal through abnormal value processing, FIR high-pass, band-pass and multi-band-pass filtering and ICA independent component analysis, so that motion artifacts are effectively inhibited, and the signal quality is improved. Then, based on pulse wave valley positioning segmentation signals, a multi-channel standardized data set is constructed; and finally, by means of an Inception-LSTM neural network fusing multi-scale feature extraction and time sequence modeling, end-to-end learning of biological features is carried out, and a high-precision identity authentication decision in a dynamic environment is realized. According to the method, the robustness is verified in seven motion state simulation scenes, and the authentication accuracy and reliability in an uncontrolled environment are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of continuous identity authentication and wearable device security, and particularly relates to a continuous identity authentication method based on multi-channel photoplethysmography (PPG) signals in an uncontrolled environment. BACKGROUND

[0002] With the popularity of smart wearable devices, users' demand for device security and privacy protection is increasing. Traditional identity authentication methods (password, pattern unlock, fingerprint recognition), although effective in certain scenarios, have obvious limitations. These methods are one-time authentication and cannot continuously verify user identity, making them vulnerable to session hijacking or context-aware attacks. When a user unlocks a device, if the device is obtained by someone else, the attacker can access sensitive data without hindrance. In addition, frequent password entry or fingerprint verification on wearable devices significantly reduces user experience, which does not meet the design concept of "unconscious interaction" for wearable devices.

[0003] To solve the above limitations of traditional identity authentication methods, continuous identity authentication technology has emerged. This technology continuously monitors users' biometric features or behavior patterns to dynamically verify user identity, thereby providing seamless security protection without active interaction. Among many biometric features, PPG signals are an ideal choice for continuous authentication due to their non-invasiveness, ease of acquisition, and unique physiological characteristics. PPG signals reflect the activity characteristics of the cardiovascular system, containing specific information about individual heart rate and vascular elasticity, and are difficult to counterfeit by external means, making them highly secure.

[0004] However, existing PPG continuous authentication systems still face significant technical challenges. First, most research relies on static PPG datasets collected in laboratory environments, i.e., signals collected while subjects remain in a stationary state (such as sitting or lying down). This data lacks dynamic changes in real-world scenarios, resulting in a significant decrease in the performance of authentication models trained based on it in actual applications. Second, PPG signals are highly susceptible to motion artifacts. When users perform daily activities (walking, running, or gesture operations), the relative displacement between the skin and the sensor produces motion artifacts, baseline drift noise, and severely obscures the effective physiological signal components. Although existing frequency domain filtering or adaptive filtering techniques can partially suppress noise, they still struggle to completely restore PPG waveform features under intense motion.

[0005] In summary, developing a PPG continuous authentication system that can adapt to dynamic environments, resist motion interference, and maintain high authentication accuracy has become a technical problem that needs to be solved in the field of wearable device security. The present application proposes an innovative solution to address these issues. SUMMARY

[0006] The application aims to provide a continuous identity authentication method based on multi-channel PPG signals in an uncontrolled environment. Through innovative signal processing procedures and neural network model construction, efficient processing of multi-channel PPG signals in an uncontrolled environment is achieved, improving the accuracy and reliability of PPG signal identity authentication technology in practical application scenarios, and solving the problem of insufficient authentication accuracy and robustness in complex environments in the prior art.

[0007] A continuous identity authentication method based on multi-channel PPG signals in an uncontrolled environment, characterized by comprising the following steps:

[0008] Step 1: Collect multi-channel PPG signals and motion sensor data under 7 motion states to form an original database;

[0009] Step 2: Data alignment and merging, aligning each channel PPG data and motion sensor data according to the sampling timestamp, and constructing a multi-channel data two-dimensional matrix;

[0010] Step 3: Abnormal value identification and processing of multi-channel PPG signals, deleting abnormal signal values;

[0011] Step 4: Multi-level filtering and ICA independent component analysis of multi-channel PPG signals to improve signal data quality;

[0012] Step 5: Based on pulse wave trough positioning, the multi-channel PPG signal sequence is segmented into fixed length segments to construct the data set of the identity authentication model;

[0013] Step 6: Establish, train and evaluate the identity authentication model based on convolutional neural network and long short-term memory network;

[0014] Specifically:

[0015] In step 1, 7 motion states are designed, including static state, arm horizontal swing, arm vertical swing, hand raising action, deep squatting and standing, walking state and running state, simulating user behavior in an uncontrolled environment. Under these 7 motion states, 4-channel PPG signals (including double-channel green light, red light and infrared light PPG signals) and motion sensor data (including three-axis acceleration data ACC and three-axis gyroscope data GYRO) are collected simultaneously using a smart watch to form an original database. Each sensor data collected here contains a sampling timestamp and corresponding signal value.

[0016] In step 2, for each sensor data collected in step 1, the PPG signal timestamp vector is specified as a reference benchmark, and the accelerometer data (ACC) and gyroscope data (GYRO) timestamps are aligned to the PPG signal sequence. At the same time, the common effective time period of each sensor data is determined, and each channel data is truncated to the common time interval. Finally, a two-dimensional matrix of multi-channel data is constructed;

[0017] In step 3, the Robust normalized value of each channel PPG signal is calculated, and the normal range of the Robust normalized absolute value is set according to the normal variation range of the PPG signal amplitude. In this invention, the normal range of the Robust normalized absolute value of the signal is set to 0.2 to 3.0: for sampling points exceeding 3.0, they are directly determined as extreme abnormal values; for sampling points less than 0.2, it is further determined whether there are 80 continuous sampling points with normalized absolute values less than 0.2, if such a continuous segment exists, this segment is determined as an extreme abnormal value segment; finally, the extreme abnormal values and extreme abnormal value segments are deleted;

[0018] In step 4, a FIR high-pass filter, a FIR band-pass filter and a FIR multi-band-pass filter are used in turn to filter and denoise the multi-channel PPG signal, and the filtered 4-channel PPG signal is combined with the accelerometer data (ACC) and the gyroscope data (GYRO) to form a mixed signal matrix. The mixed signal is subjected to ICA independent component analysis, and the component with the largest Pearson correlation coefficient with the original PPG signal is selected as the final clean signal;

[0019] In step 5, the method of the invention segments the signal sequence according to the position of the pulse wave trough, and takes 3 consecutive single-period pulse waves as a unit for signal segmentation. In the process of dynamic change of human heart rate, the length (number of sampling points) of 3 single-period pulse wave signals is also not fixed. This method normalizes the length of each signal segment to 256 sampling points, specifically, the part of the signal length greater than 256 sampling points will be truncated, and the segment of the signal length less than 256 sampling points will be filled with 0 at the end. Finally, each signal segment is labeled with a user identity tag, and a data set for identity authentication model is constructed;

[0020] In step 6, an identity authentication model is constructed based on CNN-LSTM, a one-dimensional Inception block is used to extract multi-scale waveform features of multi-channel PPG signals, a double-layer one-way LSTM is used to extract time sequence features, the features extracted by the two modules are fused in the channel dimension, then a one-dimensional convolution layer with a convolution kernel size of 3 is used to further extract features, and finally a user identity classification is performed after processing by a fully connected layer. The invention uses 10-fold cross-validation method to train the model, uses Adam optimizer to optimize the parameters, and verifies the model performance on the test set;

[0021] The method of the present application more truly simulates the uncontrolled environment by collecting data sets in 7 motion states, provides rich and varied data for model training, and improves the adaptability of the model to complex environments. The abnormal value processing, multi-level filtering denoising, ICA independent component analysis and signal segmentation method effectively improve the PPG signal quality and the usability of the data set, laying a foundation for accurate identity authentication. The model structure based on CNN+LSTM and the multi-module collaborative feature extraction can fully mine the biological feature information in the PPG signal, and significantly improve the accuracy and stability of identity authentication. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 is the overall workflow diagram of the method of the present application

[0023] Figure 2 is the Inception-LSTM identity authentication model structure diagram

[0024] Figure 3 is the Inception-LSTM identity authentication model parameter information DETAILED DESCRIPTION

[0025] The technical solution of the present application can realize automatic process operation by software technology. The technical solution of the present application will be further described in detail below in combination with the drawings and examples. Referring to Figure 1 , Figure 2 The specific steps of the embodiment of the present application are as follows:

[0026] Step 1, collect multi-channel PPG signals and motion sensor data in 7 motion states to form an original database;

[0027] Step 1.1, acquisition device: the present application uses Huawei GT2 watch to collect user wrist pulse wave data and motion sensor data. The device uses a reflective acquisition principle and integrates multiple LED light-emitting diodes and sensors on the watch, which can simultaneously collect PPG signals of four channels of double-channel green light, red light and infrared light, three-axis accelerometer data (ACC) and three-axis gyroscope data (GYRO), and the device sampling frequency is 100Hz. Before collecting data, the watch needs to be paired and connected with the data receiving terminal (Huawei mobile phone is used in this experiment) installed with the data acquisition program through Bluetooth, then the user only needs to wear the watch, start the acquisition program, and the watch will start uninterrupted acquisition of PPG signals, and the collected data will be transmitted back to the data receiving terminal in real time. The acquisition process does not interfere with the user's behavior, realizing non-invasive acquisition;

[0028] Step 1.2, personnel recruitment and preparation: the selected collection objects of the data collection experiment include 20 male users and 20 female users; the occupations cover college students, office workers, and manual laborers; the age ranges cover young, middle-aged, and old (20-65 years old); the selected collection objects are healthy and have sufficient cognitive ability to independently complete various behavioral actions in the experiment. We provide experimental training to the collection objects, which includes the use of collection equipment, demonstration of motion states, experimental procedures, and precautions;

[0029] Step 1.3, data collection execution: each collection object needs to complete data collection experiments in 7 motion states, and two sets of data are collected for each motion state, with a collection time of 5 minutes for each set. Before collecting each set of data, the collection object needs to rest for 5 minutes to allow the body and emotions to be calm and relaxed, avoiding mutual interference caused by heart rate and emotional changes in different states;

[0030] Step 2, data alignment and merging: align the PPG data and motion sensor data of each channel according to the sampling time stamp, and construct a two-dimensional matrix of multi-channel data;

[0031] The PPG signal data and motion sensor data collected by the Huawei GT2 watch do not have a one-to-one correspondence in the number of sampling points, so it is necessary to align the signal sequences of each sensor data according to the sampling time stamp and truncate the signal sequences according to the common time interval to facilitate subsequent data processing.

[0032] Step 2.1, time reference synchronization: the method of the present application specifies the PPG signal time stamp vector as the reference, and aligns the accelerometer (ACC) and gyroscope (GYRO) data time stamps to the PPG sequence;

[0033] Step 2.2, common time window truncation: determine the common effective time period of all sensor data, and truncate each channel data to the common time interval to obtain multi-channel data with the same signal sequence length and time stamp correspondence;

[0034] Step 2.3, using aligned channel data, construct a two-dimensional matrix of multi-channel data, and the data structure can be represented as:

[0035]

[0036] Where:

[0037] [t1,t2,…,t n ] T is the sampling time stamp vector, n is the number of rows of the merged data, i.e., the number of sampling points, t k is the time stamp of the kth sampling point (unit: ms), k∈[1,n];

[0038] [p 11 ,p 12 ,…,p 1n ] T It is the vector corresponding to the PPG signal value of the first green channel, p 1k It is the signal value at the kth sampling point, where k∈[1,n];

[0039] [p 21 ,p 22 ,…,p 2n ] T It is the vector corresponding to the PPG signal value of the second green channel, p 2k It is the signal value at the kth sampling point, where k∈[1,n];

[0040] [p 31 ,p 32 ,…,p 3n ] T It is the vector corresponding to the PPG signal value of the red light channel, p 3k It is the signal value at the kth sampling point, where k∈[1,n];

[0041] [p 41 ,p 42 ,…,p 4n ] T It is the vector corresponding to the PPG signal value of the infrared light channel, p 4k It is the signal value at the kth sampling point, where k∈[1,n];

[0042] The signal components of the triaxial accelerometer data on the x, y, and z axes are respectively [a] x1 ,a x2 ,…,a xn ] T , [a y1 ,a y2 ,…,a yn ] T , [a z1 ,a z2 ,…,a zn ] T , where a xk ,a yk ,a zk It is the triaxial acceleration value at the kth sampling point, where k∈[1,n];

[0043] The signal components of the three-axis gyroscope data on the x, y, and z axes are respectively [g x1 ,g x2 ,…,g xn ] T , [g y1 ,g y2..., g yn ] T , [g z1 , g z2 ..., g zn ] T wherein g xk , g yk , g zk is the three-axis gyroscope data at the kth sampling point, k [1, n] ;

[0044] Step 3, abnormal value identification and processing of the multi-channel PPG signal, deleting abnormal signal values;

[0045] In an uncontrolled environment, the acquisition device is in an unstable state, and factors such as instantaneous shaking, stumbling of the arm, and inability of the device to continuously and closely adhere to the skin will bring great motion artifact interference, so that the amplitude and frequency indicators of the PPG signal significantly deviate from the normal range, which also makes the PPG signal data processing in an uncontrolled environment face greater challenges. In order to improve the data quality, the method of the present application adopts a method based on Robust standardized value to identify abnormal sampling data.

[0046] Step 3.1, calculate the absolute value of the Robust standardized value of each channel PPG signal, the calculation formula is:

[0047]

[0048] Wherein: X is the original signal value of the PPG signal, Q1 is the first quartile of the PPG signal, Q3 is the third quartile of the PPG signal, median(X) is the median of the PPG signal value, X n is the absolute value of the PPG signal standardized value;

[0049] Step 3.2, set the normal range interval of the Robust normalized absolute value according to the normal change range of the PPG signal amplitude. The Robust normalized value being too large is generally due to the sudden shaking of the arm, and the motion artifact interference is strong, resulting in deviation; the Robust normalized value being too small is generally due to the device and the skin not being closely attached, and the signal collected is weak. In the present application, the normal range of the Robust normalized absolute value of the signal is set to 0.2 to 3.0: for the sampling points with the Robust normalized absolute value exceeding 3.0, they are directly determined as extremely large abnormal values; for the sampling points with the Robust normalized absolute value less than 0.2, it is further judged whether there are 80 continuous sampling points with the normalized absolute value less than 0.2, if there is such a continuous segment, this segment is determined as an extremely small abnormal value segment; the basis for taking 80 continuous sampling points is that: under normal circumstances, the PPG signal amplitude will rise and fall approximately periodically, showing the characteristics of wave peaks and wave troughs, and the number of sampling points of a single cycle pulse wave is about 80, if the signal amplitudes of 80 continuous sampling points are all less than 0.2, it means that the device does not collect normal PPG signal data, which can be determined as an abnormal sampling segment;

[0050] Step 3.3, for the extremely large abnormal values and the extremely small abnormal value segments identified in step 3.2, directly do the deletion processing;

[0051] Step 4, multi-level filtering denoising and ICA independent component analysis are performed on the multi-channel PPG signal to improve the signal data quality;

[0052] Step 4.1, the multi-channel PPG signal data obtained after step 3 is processed is filtered using a FIR high-pass filter, and the parameter setting is: the low-frequency cutoff frequency is 0.8 Hz, and the filter order is 301;

[0053] Step 4.2, the data obtained after step 4.1 is processed is band-pass filtered using a FIR band-pass filter, and the parameter setting is: the low-frequency cutoff frequency is 0.8 Hz, the high-frequency cutoff frequency is 5.0 Hz, and the filter order is 301;

[0054] Step 4.3, the data obtained after step 4.2 is processed is multi-band-pass filtered using a FIR multi-band-pass filter. First, the fundamental frequency f1, the second harmonic frequency f2 and the third harmonic frequency f3 of the PPG signal are identified; f1 takes the peak frequency in 0.8-5 Hz, f2 takes the peak frequency in 2f1±0.3 Hz, and f3 takes the peak frequency in 3f1±0.3 Hz. Then the frequency band bandwidth corresponding to f1, f2 and f3 is set to 0.5 Hz, the fundamental wave, the second harmonic and the third harmonic components of the PPG are extracted, and the denoised PPG signal is obtained through reconstruction;

[0055] Step 4.4, combine each channel PPG signal with ACC, GYRO data respectively to form a mixed signal matrix. Then use ICA independent component analysis technology to separate 3 independent nonlinearly correlated components from the mixed signal. Calculate the Pearson correlation coefficient between the original PPG signal and the 3 independent components respectively, and select the component with the largest correlation coefficient as the clean signal;

[0056] Step 5, based on the pulse wave valley positioning, the multi-channel PPG signal sequence is segmented into fixed length segments to construct the data set of the identity authentication model;

[0057] Step 5.1, detect the valleys of each single cycle pulse wave in the PPG signal sequence, and record the valley index, the specific steps are as follows:

[0058] Step 5.1.1, signal standardization preprocessing: perform z-score normalization operation on the 4-channel PPG signal obtained in step 4.4, the calculation method is as follows:

[0059]

[0060] Wherein: x is the PPG signal value, μ is the average value of the signal, σ is the standard deviation of the signal, z is the signal standardized value. This step eliminates the amplitude difference between channels and provides standardized input for valley detection;

[0061] Step 5.1.2, valley candidate point identification: use the peak detection algorithm based on gradient zero crossing to identify local minimum points, output all candidate valley index set T={t1,t2,…,t m}, wherein: t k k represents the serial number index of the kth candidate valley in the PPG signal sequence, m represents the total number of candidate valleys, k∈[1,m], T represents the set of candidate valley indexes;

[0062] Step 5.1.3, filter the effective valleys through the minimum interval threshold and the significance threshold. The minimum interval threshold represents the minimum time interval allowed between adjacent valleys, which is defined as half of the device sampling frequency (50 sampling points in this method). The significance threshold specifies the minimum valley depth, which is calculated as the vertical distance between the valley and its adjacent peak. The threshold is set to p=0.2, which means that the valley depth is more than 20% of the signal standard deviation. The calculation formula of the significance index is as follows:

[0063]

[0064] Wherein, v is the amplitude of the valley, l and r are the amplitudes of the left and right adjacent peaks respectively, and p represents the significance index value;

[0065] Step 5.2, multi-cycle dynamic segmentation: the method of the present application specifies the first green channel signal as the reference channel, and synchronously segments the signals from other channels based on the pulse wave trough index of the first green channel to ensure the time consistency of each signal segment in all channels. A sliding window method is used for segmentation, where each window contains three consecutive single-cycle pulse waves, and the sliding step is two cycles. Therefore, each segment contains three pulse wave cycles, and adjacent segments overlap by one cycle;

[0066] Step 5.3, signal segment length normalization: the duration of a single pulse wave cycle is about 0.6-1.0 seconds. The sampling frequency of the device is 100 Hz, so the number of sampling points for three consecutive cycles ranges from 180 to 300. In order to standardize the dimensions of the input data of the deep learning model, the length of each signal segment is normalized to 256 sampling points, specifically, the part of the signal longer than 256 sampling points will be truncated, and the segment shorter than 256 sampling points will be padded with zeros at the end;

[0067] Step 5.4, dataset construction: for the signal segment database obtained in step 5.3, each segment is labeled with a user ID tag. Thus, a dataset for identity authentication of multi-channel PPG signals in uncontrolled environments is constructed;

[0068] Step 6, construct, train and evaluate the identity authentication model based on convolutional neural network and long short-term memory network;

[0069] Step 6.1, as shown in FIG. 6, the present application realizes the extraction and classification of identity features of multi-channel PPG signals by constructing a deep neural network based on a hybrid architecture of convolutional neural network (CNN) and long short-term memory network (LSTM). The model workflow includes the following sub-steps: Figure 2

[0070] Step 6.1.1, multi-scale feature extraction: one-dimensional Inception block is used to process multi-channel PPG signals, and local features are extracted through parallel convolution paths. The Inception module contains four parallel convolution paths, each path uses different size convolution kernels for convolution operation to capture different scale features of the input data. The convolution outputs of each path are concatenated along the channel dimension to form a new feature representation containing rich scale information, which is used as the input of the subsequent module;

[0071] Step 6.1.2, a double-layer one-way long short-term memory (LSTM) network is used, the first layer of LSTM network learns the primary time sequence features of PPG signals, and the second layer of LSTM optimizes these features and further captures long-range dependencies. Its dynamic equation can be expressed as:

[0072] ​

[0073] wherein: x t represents the input vector at time step t, as the original input of the model; represents the hidden state of the first layer LSTM network at t-1, which encodes the temporal features from the historical input; represents the hidden state of the second layer LSTM network at t-1, which optimizes the primary temporal features and captures long-range dependencies; is the final hidden state at time step t, which integrates multi-scale temporal patterns as the input feature vector of the downstream classifier.

[0074] Step 6.1.3, Feature Fusion: The multi-scale features output by the Inception module and the temporal features extracted by the LSTM module are concatenated along the channel dimension. After concatenation, a one-dimensional convolution layer (with a kernel size of 3) is used to enhance the local correlation of the fused features, adjust the number of channels, and extract more refined features. Then, a max pooling operation with a kernel size of 4 and a step size of 4 is used to reduce the dimension of the feature tensor while preserving key information. After pooling, the flattened feature tensor is used as the input of the downstream classifier;

[0075] Step 6.1.4, Classifier Design: A three-layer fully connected layer (FC) architecture is used to classify the input features and obtain the identity authentication result. The number of neurons in the three-layer fully connected architecture is set to 1024, 128, and C (C is the number of user categories);

[0076] The specific parameter information of the model is presented in the accompanying Figure 3 , which is described as follows:

[0077] The identity authentication model takes a 64-batch-size, 256-length signal segment, and a 4-channel PPG signal as input (input dimension (64, 256, 4)), and outputs a prediction result for C user categories after processing by multiple modules (output dimension (64, C), C is the number of user categories to be classified). The specific logic of each module is as follows:

[0078] (1) Input layer: The model receives an initial input of dimension (64, 256, 4), where 64 represents the batch size during training, 256 is the time series length of the PPG signal segment, and 4 corresponds to the number of PPG signal channels.

[0079] (2) Data transpose layer: Adjust the dimension order of the input data (refer to the Permute(0, 2, 1) operation in the accompanying Figure 3 ) to exchange the 3rd and 2nd dimensions of the original input data to meet the input data format requirements of the model.

[0080] (3) Inception1D module: receives input of dimension (64, 4, 256), extracts multi-scale features through 4 parallel convolution paths, and finally outputs dimension (64, 64, 256). Details of each path:

[0081] Path 1: first one-dimensional convolution (Conv1d) with kernel size 1, step 1, and padding 0, mapping input from (64, 4, 256) to (64, 16, 256); then introducing nonlinearity through ReLU activation function.

[0082] Path 2: sequentially perform: ① Conv1d with kernel size 1, step 1, and padding 0, output (64, 8, 256), followed by ReLU activation; ② Conv1d with kernel size 3, step 1, and padding 1 (padding ensures output sequence length remains unchanged), converts dimension to (64, 16, 256), and then uses ReLU activation.

[0083] Path 3: sequentially perform: ① Conv1d with kernel size 1, step 1, and padding 0, output (64, 6, 256), followed by ReLU activation; ② Conv1d with kernel size 5, step 1, and padding 2 (padding maintains sequence length), output (64, 16, 256), and then ReLU activation.

[0084] Path 4: first one-dimensional maximum pooling (MaxPool1d) with kernel size 3, step 1, and padding 1, output dimension remains (64, 4, 256) (pooling preserves channel and sequence length); then Conv1d with kernel size 1, step 1, and padding 0, converts dimension to (64, 16, 256), followed by ReLU activation.

[0085] Feature concatenation: 4 paths output (all (64, 16, 256)) are concatenated along the channel dimension, finally obtaining multi-scale features of (64, 64, 256).

[0086] (4) LSTM1 module: receives input tensor of original input dimension (64, 256, 4) (initial sequence dimension without transposition), extracts primary time sequence features through a single-direction LSTM network. LSTM hidden state dimension is set to 64, and output dimension is (64, 256, 64), i.e. batch size 64, sequence length 256, and hidden feature dimension 64.

[0087] (5) LSTM2 module: receives (64, 256, 64) tensor output by LSTM1, optimizes primary time sequence features through a second layer single-direction LSTM network, and further captures long-range dependencies. Hidden state dimension remains 64, and output dimension remains (64, 256, 64).

[0088] (6) LSTM transpose layer: adjust the dimension order of LSTM output by Permute(0, 2, 1) operation: rearrange (64, 256, 64) to (64, 64, 256), align the channel dimension of time series feature with Inception1D output, and prepare for subsequent fusion.

[0089] (7) Feature concatenation layer: concatenate Inception1D output ((64, 64, 256)) and LSTM transpose layer output ((64, 64, 256)) along the channel dimension (dim = 1), and the output dimension becomes (64, 128, 256), fusing multi-scale spatial features and time series features.

[0090] (8) Fusion convolution layer: perform Conv1d operation (convolution kernel size 3, step 1, padding 1, ensure sequence length unchanged) on the concatenated (64, 128, 256) feature, compress the channel number from 128 to 64, and output (64, 64, 256); followed by ReLU activation function to introduce nonlinearity.

[0091] (9) Max pooling layer: through one-dimensional max pooling (MaxPool1d), the pooling kernel size is 4, the step is 4, and the padding is 0. The (64, 64, 256) feature is reduced in dimension: the sequence length is compressed from 256 to 64 (256 ÷ 4 = 64), and the output dimension is (64, 64, 64). It reduces the amount of calculation while retaining key features.

[0092] (10) Flatten layer: through Flatten operation, the three-dimensional feature of (64, 64, 64) is flattened into a one-dimensional vector, and the output dimension is (64, 4096) (64 x 64 x 64 = 4096), which is input for the fully connected layer.

[0093] (11) Classifier FC1 layer: through linear fully connected (Linear) layer, map 4096-dimensional input to 1024-dimensional; followed by ReLU activation function to introduce nonlinearity; then through Dropout regularization (retention probability 0.6) to suppress overfitting, and the output dimension is (64, 1024).

[0094] (12) Classifier FC2 layer: for (64, 1024) feature, map it to 128-dimensional again through Linear layer; followed by ReLU activation and Dropout regularization (retention probability 0.6), and the output dimension is (64, 128).

[0095] (13) Classifier output layer: through the last Linear layer, map the 128-dimensional feature to C-dimensional (C is the number of user categories to be classified), and the output dimension is (64, C), which is used for subsequent Softmax calculation of class probability to realize identity prediction.

[0096] In summary, the model starts from the input of PPG signal, and goes through the links of dimension adjustment, multi-scale feature extraction, time series modeling, feature fusion, dimension reduction and classification, and finally outputs the prediction result of identity category. The parameters and operations of each module cooperate to ensure the effectiveness of feature extraction and classification.

[0097] Step 6.2, model training and evaluation. The data set constructed in step 5.4 is divided into training set and test set in the ratio of 8:2. When training the model, the performance of the model is evaluated by 10-fold cross-validation. The training set is divided into 9:1, of which 9 parts are used for model training and 1 part is used for verification. The number of training rounds is set to 200 rounds, and the batch size is 64. The Adam optimizer is used, the learning rate is set to 1e-4, and the weight decay is set to 1e-5. To prevent overfitting, an early stopping strategy is set, with a patience value of 30, i.e. when the validation loss does not decrease for 30 consecutive rounds, the training is stopped. The cross-entropy loss function is used, and the formula is as follows:

[0098]

[0099] Where L represents the loss value calculated by the cross-entropy loss function; n is the number of samples; C is the number of categories of classification; y i,j is the true label of the i-th sample, represented by one-hot encoding, if sample i belongs to the j-th class, then y i,j = 1, otherwise y i,j = 0; p i,j is the probability that the model predicts that the i-th sample belongs to the j-th class.

[0100] Step 6.3, the method of the present application uses accuracy (Accuracy), precision (Precision), recall, and F1-score indicators to comprehensively evaluate the performance of the model:

[0101] Accuracy (Accuracy): represents the proportion of the number of samples correctly predicted by the model in the total number of samples, reflecting the overall prediction accuracy of the model. The calculation formula is:

[0102]

[0103] Where TP (True Positive) is the number of samples correctly predicted as positive by the model; TN (True Negative) is the number of samples correctly predicted as negative by the model; FP (False Positive) is the number of samples incorrectly predicted as positive by the model; FN (False Negative) is the number of samples incorrectly predicted as negative by the model.

[0104] Precision: Also known as precision, it refers to the proportion of samples that are actually positive examples among those predicted by the model as positive examples. This indicator reflects the accuracy of the model's prediction of positive examples, and its calculation formula is:

[0105]

[0106] Recall: Also known as recall, it refers to the proportion of samples that are actually positive examples among all samples predicted by the model as positive examples. Recall measures the model's ability to capture positive examples, and its calculation formula is:

[0107]

[0108] F1-score: It is the harmonic mean of precision and recall, which considers both precision and recall, and can more comprehensively evaluate the performance of the model. When both precision and recall are high, F1-score will also be high. Its calculation formula is:

[0109]

[0110] By comprehensively analyzing these indicators, we can more comprehensively understand the performance of the identity authentication model based on the method of the present invention in different aspects, and accurately evaluate the performance of the model.

[0111] Step 6.4, after the above steps, a pre-trained identity authentication model can be obtained, which can be deployed on related devices for real-time and continuous identity authentication tasks.

Claims

1. A continuous authentication method based on multi-channel PPG signals in an uncontrolled environment, characterized in that... Includes the following steps: Step 1: Collect multi-channel PPG signals and motion sensor data under 7 motion states to form a raw database; Step 2: Data alignment and merging. Align the PPG data and motion sensor data of each channel according to the sampling timestamp, and construct a two-dimensional matrix of multi-channel data. Step 3: Identify and process outliers in the multi-channel PPG signal, and delete outlier signal values; Step 4: Perform multi-level filtering and denoising and ICA independent component analysis on the multi-channel PPG signal to improve signal data quality; Step 5: Based on pulse wave trough localization, the multi-channel PPG signal sequence is segmented into fixed-length segments to construct the dataset for the identity authentication model; Step 6: Build, train, and evaluate an identity authentication model based on Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) network; Specifically: In step 1, seven motion states were designed, including a stationary state, horizontal arm swing, vertical arm swing, hand raising, squatting and standing, walking, and running, simulating user behavior in an uncontrolled environment. In these seven motion states, a smartwatch was used to simultaneously collect four channels of PPG signals, including dual-channel green, red, and infrared PPG signals, as well as motion sensor data, forming a raw database. Each sensor data collected here includes a sampling timestamp and the corresponding signal value. In step 2, for the sensor data collected in step 1, the PPG signal timestamp vector is specified as the reference, and the timestamps of the accelerometer data ACC and the gyroscope data GYRO are aligned to the PPG signal sequence; at the same time, the common effective time period of each sensor data is determined, and the data of each channel is extracted to the common time interval; finally, a multi-channel data two-dimensional matrix is ​​constructed. In step 3, the absolute value of the Robust normalized value of each channel PPG signal is calculated, and the normal range of the absolute value of the Robust normalized value is set according to the normal range of PPG signal amplitude variation. The normal range for the Robust standardized absolute value of the signal is set to 0.2 to 3.0: sampling points exceeding 3.0 are directly identified as maximum outliers; sampling points less than 0.2 are further checked to see if there are 80 consecutive sampling points with standardized absolute values ​​less than 0.

2. If such a continuous segment exists, this segment will be identified as a minimum outlier segment; finally, maximum and minimum outlier segments are deleted. In step 4, FIR high-pass filter, FIR band-pass filter and FIR multi-band-pass filter are used in sequence to filter and denoise the multi-channel PPG signal. The filtered 4-channel PPG signal is combined with accelerometer data ACC and gyroscope data GYRO to form a mixed signal matrix. ICA independent component analysis is performed on the mixed signal, and the component with the largest Pearson correlation coefficient with the original PPG signal is selected as the final clean signal. In step 5, the signal sequence is segmented based on the trough position of the pulse wave signal, with three consecutive single-cycle pulse waves as a unit for signal segmentation; the length of each signal segment is normalized to 256 sampling points, specifically, the portion of the signal length greater than 256 sampling points is truncated, and segments with a signal length less than 256 sampling points are padded with zeros at the end; finally, each signal segment is labeled with a user identity tag to construct the dataset for the identity authentication model. In step 6, an identity authentication model is constructed based on CNN-LSTM. A one-dimensional Inception block is used to extract multi-scale waveform features of multi-channel PPG signals, and a two-layer unidirectional LSTM network is used to extract temporal features. The features extracted by the two modules are fused in the channel dimension, and then further features are extracted through a one-dimensional convolutional layer with a kernel size of 3. Finally, after processing by a fully connected layer, user identity is classified. The model is trained using the 10-fold cross-validation method, and the parameters are optimized using the Adam optimizer.

2. The continuous authentication method based on multi-channel PPG signals in an uncontrolled environment according to claim 1, characterized in that: The smartwatch described in step 1 integrates multiple sensors that synchronously sample at a sampling frequency of 100Hz.

3. The continuous authentication method based on multi-channel PPG signals in an uncontrolled environment according to claim 1, characterized in that: The Inception-LSTM authentication model employs a one-dimensional Inception module containing four parallel convolutional paths. Each path uses convolutional kernels of different sizes to perform convolution operations, capturing features at different scales of the input data. Simultaneously, a two-layer unidirectional LSTM network is used to learn the temporal features of the PPG signal. The multi-scale features output by the Inception module and the temporal features extracted by the LSTM module are concatenated along the channel dimension for feature fusion.

4. The continuous authentication method based on multi-channel PPG signals in an uncontrolled environment according to claim 1, characterized in that, The specific parameter information of the identity authentication model is as follows: This identity authentication model takes 64 batches of signal segments, each 256 segments long, and 4-channel PPG signals as input; the input dimension is (64, 256, 4). After multi-module collaborative processing, the final output is the prediction result for user category C corresponding to the 64 batches, with an output dimension of (64, C), where C is the number of user categories to be classified, as follows: (1) Input layer: The model receives an initial input tensor of dimension (64,256,4), where 64 represents the batch size during training, 256 is the time series length of the PPG signal segment, and 4 corresponds to the number of channels of the PPG signal. (2) Data transpose layer: Adjust the order of the dimensions of the input data, and swap the order of the third and second dimensions of the original input data to meet the model’s requirements for the input data format; (3) Inception1D module: Receives input of dimension (64, 4, 256), extracts multi-scale features through 4 parallel convolutional paths, and finally outputs dimension (64, 64, 256). Details of each path are as follows: Path 1: First, a one-dimensional convolution is performed with a kernel size of 1, a stride of 1, and padding of 0, mapping the input from (64,4,256) to (64,16,256); then, nonlinearity is introduced through the ReLU activation function. Path 2: Execute the following in sequence: ① Conv1d with kernel size of 1, stride of 1, and padding of 0, outputting (64, 8, 256), followed by ReLU activation; ② Conv1d with kernel size of 3, stride of 1, and padding of 1, transforming the dimensions to (64, 16, 256), and then using ReLU activation. Path 3: Execute in sequence: ① Conv1d with kernel size 1, stride 1, and padding 0, output (64,6,256), followed by ReLU activation; ② Conv1d with kernel size 5, stride 1, and padding 2, output (64,16,256), followed by ReLU activation; Path 4: First, perform one-dimensional max pooling (MaxPool1d) with a kernel size of 3, stride of 1, and padding of 1, so the output dimension is still (64, 4, 256); then, use Conv1d with a kernel size of 1, stride of 1, and padding of 0 to transform the dimension to (64, 16, 256), followed by ReLU activation. Feature concatenation: The outputs of the four paths are all (64, 16, 256) concatenated along the channel dimension to finally obtain a multi-scale feature of (64, 64, 256); (4) LSTM1 module: Receives the original input dimension (64,256,4) tensor and extracts primary temporal features through a unidirectional LSTM network; The LSTM hidden state dimension is set to 64, and the output dimension is (64, 256, 64), which means the batch size is 64, the sequence length is 256, and the hidden feature dimension is 64. (5) LSTM2 module: Receives the (64,256,64) tensor output by LSTM1, optimizes the primary temporal features through the second layer of unidirectional LSTM network, and further captures long-range dependencies; the hidden state dimension is still 64, and the output dimension remains (64,256,64). (6) LSTM transpose layer: Adjust the dimensional order of the LSTM output through the Permute(0,2,1) operation: rearrange (64,256,64) into (64,64,256) to align the channel dimension of the temporal features with the Inception1D output; (7) Feature splicing layer: The output of Inception1D ((64,64,256)) and the output of LSTM transpose layer ((64,64,256)) are spliced ​​along the channel dimension dim=1, and the output dimension becomes (64,128,256), which integrates multi-scale spatial features and temporal features. (8) Fusion convolutional layer: Perform Conv1d operation on the concatenated (64,128,256) features with kernel size 3, stride 1, and padding 1 to keep the sequence length unchanged, compress the number of channels from 128 to 64, and output (64,64,256); then follow with ReLU activation function to introduce non-linearity; (9) Max Pooling Layer: Through one-dimensional max pooling (MaxPool1d), with a pooling kernel size of 4, a stride of 4, and padding of 0, the dimensionality of (64,64,256) features is reduced: the sequence length is compressed from 256 to 64, and the output dimension is (64,64,64). (10) Flattening layer: The three-dimensional features of (64,64,64) are flattened into a one-dimensional vector through the Flatten operation, and the output dimension (64,4096) is prepared as input for the fully connected layer. (11) Classifier FC1 layer: The 4096-dimensional input is mapped to 1024-dimensional through a linear fully connected layer; then the ReLU activation function is used to introduce nonlinearity; then Dropout regularization is used to retain the probability of 0.6 to suppress overfitting, and the output dimension is (64, 1024). (12) Classifier FC2 layer: For (64,1024) features, the linear layer is used to map them to 128 dimensions again; Followed by ReLU activation and Dropout regularization, retain probability 0.6, output dimension (64,128); (13) Classifier output layer: The 128-dimensional features are mapped to C dimensions through the last Linear layer, where C is the number of user categories to be classified. The output dimension is (64, C), which is used for subsequent Softmax calculation of category probabilities to achieve identity prediction.

Images