PPG signal cross-period identity recognition method and device based on domain self-adaption

Through the domain-adaptive DANN network, combined with deep neural network, common characteristics of PPG signals are extracted in different periods, and the accuracy of PPG signal identity recognition is reduced in different periods, achieving high accuracy and system availability of cross-period identity recognition .

CN119071032AActive Publication Date: 2024-12-03SOUTHEAST UNIV
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202411090265.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-09
Publication Date
2024-12-03
Estimated Expiration
2044-08-09

AI Technical Summary

Technical Problem

The existing identity recognition technology based on PPG signals has caused the intrinsic changes in user PPG signals in different periods to reduce the accuracy of identity recognition, and the system availability is greatly reduced.

Method used

DANN network adopts a domain adaptive design, combined with deep neural network, collects fingertip videos through a smartphone camera, performs signal filtering, standardization and window processing, and uses one-dimensional convolutional neural network, bidirectional gating logic unit and self-attention mechanism to build a domain-adversarial neural network to extract common features from different periods.

Benefits of technology

It effectively alleviates the problem of the decline in identity recognition accuracy in cross-period scenarios, ensures the accuracy of identification results and the long-term availability of the system, and can be expanded to multiple periods of data, with better performance than traditional manual feature extraction and neural network methods.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119071032A_ABST
    Figure CN119071032A_ABST
Patent Text Reader

Abstract

The invention discloses a PPG signal cross-period identity recognition method and device based on domain self-adaption, and the method comprises the steps: collecting a fingertip video through an intelligent mobile device, obtaining a user PPG signal, and enabling the collected PPG signal to serve as the input of a deep neural network, and completing an identity recognition task. In order to ensure the identity recognition accuracy of PPG signals in a cross-period scene, a domain adversarial neural network DANN is designed in combination with domain self-adaption on the basis of a deep neural network. The advantages of convenience and rapidness of the mobile terminal and high accuracy of the neural network can be played, and safe and convenient good experience is provided for the user.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of PPG signals, and in particular to a method and device for cross-period identity recognition of PPG signals based on domain adaptation. Background Art

[0002] In today's society, with the continuous improvement of information technology, the information and property security issues of individuals, enterprises and even government agencies have become increasingly prominent. As the first barrier to isolate risks, identity recognition technology is also facing higher requirements and more challenges. In order to adapt to the development needs of the current information age, identity recognition technology based on photoplethysmographic (PPG) has become a research hotspot in the current field of identity recognition due to its advantages of being difficult to forge and steal and requiring liveness verification.

[0003] As an identification method that is more in line with the background of the times, compared with previous generations of identification technologies, PPG signals are more concealed and difficult to forge and steal. They require liveness verification during identification, so they have better security performance. Compared with ECG signals of the same generation, they are easy to collect and do not require high-end and expensive equipment. Therefore, they have broader application prospects and greater demand for use. In recent years, they have been regarded as a research hotspot. In terms of application scenarios, PPG signals can meet the smart medical scenarios of patient identification and personalized diagnosis, financial business scenarios of payment security and property protection, and work scenarios of corporate employment and clocking in. But at the same time, the current identity recognition task based on PPG signals also has defects: as a data-driven task, the performance of the identity recognition task based on PPG signals will be highly dependent on the input data. The inherent changes in the user's PPG signals in different periods will inevitably lead to a decrease in the accuracy of identity recognition, and the system availability will be greatly reduced. For this reason, it is necessary to design a method that can meet the needs of cross-period PPG signal identity recognition. This method can extract the common features of the user's PPG signals in different periods, and the extracted common features can not only remain stable in different periods to meet the needs of long-term authentication, but also ensure the distinguishability of the input features of the identity recognition system and the accuracy of the recognition results. Summary of the invention

[0004] Technical issues:

[0005] The purpose of the present invention is to provide a cross-period identity recognition method based on PPG signals to solve the problem that the existing deep neural network has a reduced accuracy rate of identity recognition and a greatly reduced system availability due to the inherent changes in the user's PPG signals in different periods. Based on the deep neural network, the DANN network is designed in combination with domain adaptation to improve the accuracy of identity recognition in cross-period scenarios.

[0006] Technical solution:

[0007] In view of the above defects or improvement needs of the prior art, the present invention provides a method and device for cross-period identity recognition of PPG signals based on domain adaptation, comprising the following steps:

[0008] S1: Collect fingertip videos through the smartphone camera and upload the fingertip videos to the cloud server to complete the initial data collection;

[0009] S2: Extract the PPG signal with the best signal quality based on the fingertip video; first filter it with a second-order Butterworth filter of 0.5-20Hz to reduce the noise interference of motion artifacts and power frequency interference while retaining the main frequency range of the PPG signal; then standardize it with the Z-score so that the mean of the PPG signal becomes 0 and the standard deviation becomes 1; the standardized PPG signal is windowed with the main peak as the starting point, the size of the signal window is 100, and 1s is selected as the signal window duration, that is, if the sampling frequency is 100Hz, the PPG signal contains 100 data points, and the PPG signal segments that do not meet 1s will be discarded;

[0010] S3: The PPG signal is used as the input of the deep neural network DNN for training. The DNN includes a one-dimensional convolutional neural network 1D CNN, a bidirectional gated logic unit BiGRU, a self-attention mechanism Self-Attention and a fully connected layer;

[0011] S4: The PPG signal identity recognition task based on the deep neural network DNN has the problem that the accuracy of identity recognition decreases due to the intrinsic changes of the PPG signal in the cross-period scenario. In order to better learn the common features of the PPG signal in different periods, based on the deep neural network DNN in step S3, a domain adversarial neural network DANN is built in combination with domain adaptation; the DANN network consists of four parts, namely, the feature extractor G f , label classifier G y , domain discriminator G d and gradient reversal layer GRL; the DNN network in step S3 is partially reused as the feature extractor of the DANN network except for the fully connected layer, and a fully connected layer is added on the basis of the feature extractor to design the label classifier G y and domain discriminator G d ;

[0012] S5: The identity recognition task is completed through the DANN network, and the final identity recognition result is transmitted back to the user App terminal for continuing the downstream related tasks of identity recognition.

[0013] Furthermore, the PPG signal with the best signal quality performance was obtained by comparison: in the color space channel, the PPG signal quality performance of the R, G, B channels of the RGB space, the Y channel of the YIQ, and the V channel of the HSV space were compared; in the signal extraction area, the image of the input fingertip video was divided into multiple ROI video sub-blocks and the PPG signal quality performance was compared on the corresponding video sub-blocks respectively; the signal quality performance was measured by the amplitude change AC and the signal-to-noise ratio SNR.

[0014]

[0015] Where X j represents the j-th signal window PPG signal segment, the signal window size is set to 30, n represents the total number of current PPG signal windows, and the last signal window whose length does not meet 30 will be discarded;

[0016] SNR = 10log(P s / P n ),

[0017] Where P s and P n Represent the power P of the signal and noise respectively. For a signal x(n) with a signal length of n, the calculation formula of the power P is

[0018] Furthermore, the deep neural network DNN includes a one-dimensional convolutional neural network layer 1D CNN, a bidirectional gated logic unit BiGRU, a self-attention mechanism Self-Attention and a fully connected layer; the DNN input PPG signal dimension is 100*1, the number of convolution layers of the one-dimensional convolutional neural network layer is 1, the convolution layer kernel size is 3*1, the number of convolution kernels is 64, the Dropout is 20%, the number of bidirectional GRU layers is 1, the GRU hidden state is 256, the input of the self-attention mechanism is 128*2, the input and output of the fully connected layer is 256*1 / number of certified persons*1, the weight decay term is 0.001, and 0.01 is used for L2 regularization.

[0019] Furthermore, combined with the domain adaptive domain adversarial neural network DANN, a total of feature extractors G d , label classifier G y , domain discriminator G d and gradient reversal layer GRL; reuse the network content of the DNN network except the fully connected layer as the feature extractor of the DANN network, and design the label classifier G through the fully connected layer respectively y , domain discriminator G d; The label classifier includes 2 layers of fully connected FC layers, the input / output of the first FC layer is 512*1 / 128*1 respectively, followed by a Dropout layer with a random inactivation rate of 50%, and the input / output of the second FC layer is 128*1 / number of classified people*1 respectively; the domain discriminator also includes 2 layers of FC layers, the input / output of the first FC layer is 56*1 / 256*1 respectively, and the input / output of the second FC layer is 256*1 / 2*1 respectively; the gradient reversal layer is used to control the reversal strength, and its parameter is controlled by λ, and the size settings include 1, 1.5, and 2.

[0020] Furthermore, combined with the domain adaptive domain adversarial neural network DANN, the objective function includes the label classification loss L ce and domain classification loss L dann , label classification loss L ce Calculated by the cross entropy loss function;

[0021]

[0022] Where p(x i ) represents the probability that the true label is of this category (0 or 1); q(x i ) represents the probability of the model predicting the category, and m represents the total number of all categories.

[0023]

[0024] Where x s and x t Represent the data samples of the source domain and the target domain respectively, p s (x) and p t (x) represents the marginal probability distribution of source domain and target domain data samples, θ f Represents the feature extractor G f Parameter, θ d Representative domain discriminator G d Parameter, E represents the expectation of the corresponding project.

[0025] A domain-adaptive PPG signal cross-period identity recognition device is characterized in that it includes a smart phone and a cloud server, the camera of the smart phone completes the fingertip video recording task in claim 1, and uploads the collected fingertip video to the cloud server; the cloud server implements the contents of steps S2-S5 in claims 1-5.

[0026] Beneficial effects:

[0027] By combining domain adaptation technology, the PPG signal of the current period is regarded as the source domain, and the PPG signal of the future or another period is regarded as the target domain. The feature mapping of the PPG signals of different periods is aligned, and then the optimal decision boundary is re-divided, which can effectively alleviate the problem of decreased accuracy of identity recognition in cross-period scenes. The model combined with domain adaptation can not only perform well in the environment or scene of the current period, but also maintain considerable performance in the environment and scene of the future or another period; and because the domain adaptation technology fully considers the changes in the characteristics of the PPG signals in different periods, the model can better capture the characteristics of the PPG signals in different periods, even if the data has not appeared in the training process, the recognition performance of the model will be better; secondly, the current public data sets used for cross-period identity recognition only contain PPG signals of two different periods, and domain adaptation can expand the identity recognition task to two or more periods of data. Different from the traditional method of manually extracting features, it can avoid complex feature engineering construction, and has better recognition performance than the existing neural network method. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 DANN network structure diagram in the present invention.

[0029] Figure 2 Schematic diagram of fingertip video acquisition based on a mobile phone in the present invention.

[0030] Figure 3 Schematic diagram of video sub-block division of fingertip video in the present invention

[0031] Figure 4 AC heat map of PPG signals of different video sub-blocks in the present invention.

[0032] Figure 5 SNR heat map of PPG signals of different video sub-blocks in the present invention.

[0033] Figure 6 Visualization result diagram of the DANN network ONNX model of the present invention.

[0034] Figure 7 t-SNE diagram of PPG signals of 31 users of the DANN network in different periods of the present invention.

[0035] Figure 8 Schematic diagram of the mobile phone Android App interface in the present invention. Specific implementation methods

[0037] The present invention is further described in detail below in conjunction with the accompanying drawings and implementation examples. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0038] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0039] Example 1

[0040] This embodiment provides a method for cross-period identity recognition of PPG signals based on domain adaptation, and the DANN neural network structure used is as follows: Figure 1 As shown, the following steps are included:

[0041] Step 1: Collect fingertip video through the smartphone camera, such as Figure 2 As shown, the fingertip video is uploaded to the cloud server to complete the initial data collection;

[0042] Step 2: Complete the PPG signal extraction of the G channel on the specific video sub-block of the fingertip video. The video sub-block division information is as follows: Figure 3 As shown; a second-order Butterworth filter with a frequency of 0.5 to 20 Hz is used for filtering, and then the signal is standardized by Z-score; the standardized PPG signal is windowed with the main peak as the starting point, and the size of the signal window is 100. Figure 4 and Figure 5 The AC and SNR heat map distributions of each video sub-block are shown respectively. On this basis, the PPG signal is extracted only on the union of video sub-blocks 9 and 10, so that the PPG signal quality is optimal;

[0043] Step 3: Use the PPG signal as the input of the deep neural network DNN for training. The deep neural network DNN includes a one-dimensional convolutional neural network 1D CNN, a bidirectional gated logic unit BiGRU, a self-attention mechanism Self-Attention and a fully connected layer;

[0044] Step 4: Combine domain adaptation to build a domain adversarial neural network DANN. Figure 6 The visualization results of the DANN network ONNX model are shown. The network can effectively learn the common features of PPG signals at different stages. The DANN network consists of four parts: feature extractor G f , label classifier G y , domain discriminator G d and gradient reversal layer GRL; the DNN network in step 3 is partially reused as the feature extractor of the DANN network except for the fully connected layer, and a fully connected layer is added on the basis of the feature extractor to design the label classifier Gy and domain discriminator G d The t-sne results of the 31 users in the network at different times are as follows Figure 7 As shown, the feature distribution of the same user has the same color, and the different shapes represent the PPG signal features of different periods.

[0045] Step 5: The identity recognition task is completed through the DANN network, and the final identity recognition result is transmitted back to the user's App terminal. The relevant interface of the App terminal is as follows Figure 8 , used to continue downstream identity recognition related tasks.

[0046] The exemplary embodiments described above only express several embodiments of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A method for cross-period identity recognition of PPG signals based on domain adaptation, characterized in that: The following steps are involved: S1: Collect fingertip videos through the smartphone camera and upload the fingertip videos to the cloud server to complete the initial data collection; S2: Extract the PPG signal with the best signal quality based on the fingertip video; first filter it with a second-order Butterworth filter of 0.5-20Hz to reduce the noise interference of motion artifacts and power frequency interference while retaining the main frequency range of the PPG signal; then standardize it with the Z-score so that the mean of the PPG signal becomes 0 and the standard deviation becomes 1; the standardized PPG signal is windowed with the main peak as the starting point, the size of the signal window is 100, and 1s is selected as the signal window duration, that is, if the sampling frequency is 100Hz, the PPG signal contains 100 data points, and the PPG signal segments that do not meet 1s will be discarded; S3: The PPG signal is used as the input of the deep neural network DNN for training. The DNN includes a one-dimensional convolutional neural network 1DCNN, a bidirectional gated logic unit BiGRU, a self-attention mechanism Self-Attention and a fully connected layer; S4: The PPG signal identity recognition task based on the deep neural network DNN has the problem of reduced identity recognition accuracy due to the intrinsic changes of the PPG signal in the cross-period scenario. In order to better learn the common features of the PPG signal in different periods, based on the deep neural network DNN in step S3, a domain adversarial neural network DANN is built in combination with domain adaptation; the DANN network consists of four parts, namely, the feature extractor G f , label classifier G y , domain discriminator G d and gradient reversal layer GRL; the DNN network in step S3 is partially reused as the feature extractor of the DANN network except for the fully connected layer, and a fully connected layer is added on the basis of the feature extractor to design the label classifier G y and domain discriminator G d ; S5: The identity recognition task is completed through the DANN network, and the final identity recognition result is transmitted back to the user App terminal for continuing the downstream related tasks of identity recognition.

2. The method for cross-period identity recognition based on domain adaptation of PPG signals according to claim 1, characterized in that: The PPG signal with the best signal quality performance was obtained by comparison: in the color space channel, the PPG signal quality performance of the R, G, B channels of the RGB space, the Y channel of the YIQ, and the V channel of the HSV space was compared; in the signal extraction area, the image of the input fingertip video was divided into multiple ROI video sub-blocks and the PPG signal quality performance was compared in the corresponding video sub-blocks respectively; the signal quality performance was measured by the amplitude change AC and the signal-to-noise ratio SNR; Where X j represents the j-th signal window PPG signal segment, the signal window size is set to 30, n represents the total number of current PPG signal windows, and the last signal window whose length does not meet 30 will be discarded; SNR=10log(P s / P n ), Where P s and P n Represent the power P of the signal and noise respectively. For a signal x(n) with a signal length of n, the calculation formula of the power P is 3. The method for cross-period identity recognition based on domain adaptation of PPG signals according to claim 1, characterized in that: The deep neural network DNN includes a one-dimensional convolutional neural network layer 1D CNN, a bidirectional gated logic unit BiGRU, a self-attention mechanism Self-Attention and a fully connected layer; the DNN input PPG signal dimension is 100*1, the number of convolution layers of the one-dimensional convolutional neural network layer is 1, the convolution layer kernel size is 3*1, the number of convolution kernels is 64, the Dropout is 20%, the number of bidirectional GRU layers is 1, the GRU hidden state is 256, the input of the self-attention mechanism is 128*2, the input and output of the fully connected layer is 256*1 / number of certified people*1, the weight decay term is 0.001, and 0.01 is used for L2 regularization.

4. The method for cross-period identity recognition based on domain adaptation of PPG signals according to claim 1, characterized in that: Combined with the domain adaptive domain adversarial neural network DANN, a total of feature extractors G d , label classifier G y , domain discriminator G d and gradient reversal layer GRL; reuse the network content of the DNN network except the fully connected layer as the feature extractor of the DANN network, and design the label classifier G through the fully connected layer respectively y , domain discriminator G d ; The label classifier includes 2 layers of fully connected FC layers, the input / output of the first FC layer is 512*1 / 128*1 respectively, followed by a Dropout layer with a random inactivation rate of 50%, and the input / output of the second FC layer is 128*1 / number of classified people*1 respectively; the domain discriminator also includes 2 layers of FC layers, the input / output of the first FC layer is 56*1 / 256*1 respectively, and the input / output of the second FC layer is 256*1 / 2*1 respectively; the gradient reversal layer is used to control the reversal strength, and its parameter is controlled by λ, and the size settings include 1, 1.5, and 2.

5. The method for cross-period identity recognition based on domain adaptation of PPG signals according to claim 4, characterized in that: Combined with the domain adaptive domain adversarial neural network DANN, the objective function includes the label classification loss L ce and domain classification loss L dann , label classification loss L ce Calculated by the cross entropy loss function; Where p(x i ) represents the probability that the true label is of this category, which is 0 or 1; q(x i ) represents the probability of the model predicting this category, and m represents the total number of all categories; Where x s and x t Represent the data samples of the source domain and the target domain respectively, p s (x) and p t (x) represents the marginal probability distribution of source domain and target domain data samples, θ f Represents the feature extractor G f Parameter, θ d Representative domain discriminator G d Parameter, E represents the expectation of the corresponding item.

6. A PPG signal cross-period identity recognition device based on domain adaptation, characterized in that: It includes a smart phone and a cloud server, wherein the camera of the smart phone completes the fingertip video recording task in claim 1 and uploads the collected fingertip video to the cloud server; the cloud server implements the contents of steps S2-S5 in claims 1-5.

Citation Information

Patent Citations

  • Attention mechanism-based adversarial domain adaptive electroencephalogram emotion recognition method in cross-user scene

    CN114492513A

  • Cross-time individual identification method and system based on electroencephalogram signal deep learning

    CN114912492A

  • PPG identity verification method based on triple network

    CN118114090A

  • Side channel attack method based on domain adversarial neural network

    CN118249981A

  • Proactively authenticating users with biometric devices based on photoplethysmograms

    WO2023073177A1