A deep learning-based live face recognition and encrypted transmission method

Abstract

This invention relates to the interdisciplinary field of image analysis and information security, specifically a deep learning-based method for live face recognition and encrypted transmission. The method involves acquiring facial video frames, constructing a spatiotemporal graph convolutional network to extract subtle color change features of facial pixels during the heartbeat cycle, and obtaining a spatiotemporal frequency domain response map. This map is then input into a dual-branch deep network. The first branch outputs the liveness detection result, while the second branch maps the map to a biometric vector and quantizes it into a dynamic symmetric key for encrypted data transmission. In cases of non-liveness, a random perturbation vector is output to replace the dynamic symmetric key, blocking the encryption link. This invention imbues the encryption key with liveness physiological dependence, solving the problem of key leakage caused by the disconnect between liveness recognition and encryption, and blocking data theft paths in non-liveness states.

Description

Technical Field

[0001] This invention relates to the intersection of image analysis and information security, and specifically to a method for live face recognition and encrypted transmission based on deep learning. Background Technology

[0002] In existing live face recognition and encrypted transmission schemes, liveness detection and encrypted transmission are performed as two independent steps. In the liveness detection phase, the system acquires facial images or video streams and uses neural networks to extract facial texture or motion features to distinguish between real faces and forged entities. In the encrypted transmission phase, the system uses an independent key negotiation protocol to generate a session key, encrypting the recognition results and business data. The receiving end uses the negotiated key to decrypt the data. The liveness detection result serves only as a Boolean value to determine whether passage is allowed; there is no data-level correlation between the encryption key generation process and the liveness physiological characteristics.

[0003] In existing technologies, liveness detection and encrypted transmission are decoupled, and the generation of encryption keys is independent of the physiological characteristics of the live body, resulting in a disconnect between the key and the real identity. If the key is illegally intercepted during transmission or storage, attackers can use the intercepted key to decrypt data or forge encrypted data without the presence or participation of a live person, posing security risks of data leakage and identity forgery. Summary of the Invention

[0004] The purpose of this invention is to provide a deep learning-based method for live face recognition and encrypted transmission, which can effectively solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A deep learning-based method for live face recognition and encrypted transmission includes: acquiring a continuous sequence of face video frames, constructing a spatiotemporal graph convolutional network, performing temporal frequency domain transformation on the pixel values ​​of the same node in different video frames along the time dimension to extract pulse wave frequency band signals, and performing graph convolution aggregation operation on the pulse wave frequency band signals along the spatial dimension to generate a spatiotemporal frequency domain response map. The spatiotemporal frequency domain response map is input into a dual-branch deep network, and the first branch classifier of the dual-branch deep network is used to classify the spatiotemporal frequency domain response map and output the liveness detection result. Simultaneously, the spatiotemporal frequency domain response spectrum is input into the multilayer fully connected layer through the second branch of the dual-branch deep network and dimensionality reduction mapping is transformed into a fixed-length floating-point biometric vector. Then, local sensitive hashing is performed on the floating-point biometric vector to convert it into a binary bit string. Finally, a dynamic symmetric key is generated through the key derivation function. During the encrypted transmission phase, the identification result and business data to be transmitted are encrypted using the dynamic symmetric key to generate ciphertext data and send it to the receiving end. The receiving end decrypts the ciphertext data using the biometric vector that has been securely exchanged in advance. When the liveness detection result is non-liveness, the second branch of the dual-branch deep network outputs a random perturbation vector to replace the dynamic symmetric key, thereby blocking the encrypted link.

[0006] Preferably, the step of constructing a spatiotemporal graph convolutional network to extract the subtle color change features of facial region pixels in the face video frame sequence as the heartbeat cycle generates, and obtaining a spatiotemporal frequency domain response map, includes: performing facial key point detection on the face video frame sequence, and constructing a facial topology graph structure based on the detected facial key points, wherein the nodes of the facial topology graph structure correspond to the facial spatial location, and the edges of the facial topology graph structure correspond to the facial spatial adjacency relationship; The facial topology graph structure is input into the spatiotemporal graph convolutional network. The pixel values ​​of the same node in the facial topology graph structure in different facial video frame sequences are transformed along the time dimension to extract the pulse wave frequency band signal corresponding to the node. A graph convolution aggregation operation is performed on the pulse wave frequency band signal along the spatial dimension on the edges of the facial topology structure. The aggregated frequency domain features are then subjected to spectral reconstruction to generate the spatiotemporal frequency domain response map.

[0007] Preferably, the step of classifying the spatiotemporal frequency domain response map by the first branch classifier of the dual-branch deep network and outputting the liveness detection result includes: inputting the spatiotemporal frequency domain response map into the first branch classifier and extracting the frequency distribution features and energy concentration features in the spatiotemporal frequency domain response map; The frequency distribution feature and the energy concentration feature are concatenated, and the concatenated joint feature is input into a probability discriminant function based on a Gaussian mixture model to calculate the posterior probability that the joint feature belongs to the distribution of real living physiological signals. When the posterior probability is greater than a preset probability threshold, the liveness determination result representing a live body is output; when the posterior probability is less than or equal to the preset probability threshold, the liveness determination result representing a non-live body is output.

[0008] Preferably, the step of mapping the spatiotemporal frequency domain response map into a fixed-length biometric vector through the second branch of the dual-branch deep network, and quantizing the biometric vector into a dynamic symmetric key, includes: inputting the spatiotemporal frequency domain response map into a multilayer fully connected layer of the second branch, and dimensionality-reducing the spatiotemporal frequency domain response map into the fixed-length floating-point biometric vector; Perform a locality-sensitive hash operation on the floating-point biometric vector to convert the floating-point biometric vector into a fixed-length binary bit string; Based on the binary bit string, the dynamic symmetric key is generated by calculating the national cryptographic key derivation function, and the dynamic symmetric key is dynamically updated as the spatiotemporal frequency domain response spectrum evolves over time.

[0009] Preferably, the step of the receiving end decrypting the encrypted data using the pre-securely exchanged biometric vector includes: in the initial registration stage, the sending end collects a registration video frame sequence and generates a registration biometric vector, encrypts the registration biometric vector using an asymmetric encryption algorithm, and then sends it to the receiving end; During the encrypted transmission phase, the receiving end uses a private key to decrypt and obtain the registered biometric vector, and generates a registration dynamic symmetric key based on the registered biometric vector; The receiving end receives the ciphertext data and uses the registered dynamic symmetric key to decrypt and verify the ciphertext data. If the decryption is successful, the identification result is confirmed to be valid; if the decryption fails, the ciphertext data is discarded.

[0010] Preferably, when the liveness detection result is non-live, the step of replacing the dynamic symmetric key with a random perturbation vector output by the second branch of the dual-branch deep network to block the encrypted link includes: when the liveness detection result is non-live, triggering the perturbation generation mechanism of the second branch of the dual-branch deep network; The random perturbation vector is generated based on the current system timestamp and the hardware random number generator; The dynamic symmetric key is replaced by the random perturbation vector. The identification result and the business data to be transmitted are encrypted using the random perturbation vector to generate an undecryptable interference ciphertext, which is then sent to the receiving end. This causes the receiving end to fail to decrypt the biometric vector based on the pre-securely exchanged biometric vector, thereby blocking the encrypted link.

[0011] Preferably, the step of performing a time-series frequency domain transformation on the pixel values ​​of the same node in the facial topology structure in different facial video frame sequences along the time dimension to extract the pulse wave frequency band signal corresponding to the node includes: performing a fast Fourier transform on the pixel values ​​of the same node along the time dimension to obtain the spectral distribution of the same node; Locate the target frequency band corresponding to the cardiac cycle in the spectrum distribution, and extract the original signal in the target frequency band as the initial pulse wave signal; The initial pulse wave signal is subjected to frequency domain noise reduction using an adaptive Wiener filter, and the signal whose signal-to-noise ratio meets the preset condition after noise reduction is determined as the pulse wave frequency band signal.

[0012] Preferably, after calculating the posterior probability that the joint features belong to the distribution of real living physiological signals, the method further includes: obtaining multiple joint features corresponding to multiple face video frame sequences within a continuous time window; Calculate the temporal cosine similarity between the joint features of adjacent time slices, and extract the difference absolute value features of the joint features of adjacent time slices; The posterior probability, the temporal cosine similarity, and the absolute difference feature are weighted and fused to obtain the comprehensive liveness confidence. When the overall liveness confidence score is greater than a preset overall threshold, the liveness determination result representing a live body is output; when the overall liveness confidence score is less than or equal to the preset overall threshold, the liveness determination result representing a non-live body is output.

[0013] Preferably, the step of performing locality-sensitive hashing on the floating-point biometric vector to convert the floating-point biometric vector into a fixed-length binary bit string includes: generating multiple sets of random hyperplane hash function clusters; The floating-point biometric vector is subjected to a fragmented hash mapping using the multiple sets of random hyperplane hash function clusters to obtain multiple local binary substrings. The multiple local binary substrings are concatenated into the binary bit string in a preset order; The binary bit string is input into a key derivation function based on a hash message authentication code, and the current timestamp salt value is added to generate the dynamic symmetric key.

[0014] Preferably, in the initial registration stage, the step of the sending end collecting the registration video frame sequence and generating the registration biometric vector, and encrypting the registration biometric vector using an asymmetric encryption algorithm before sending it to the receiving end includes: in the initial registration stage, the sending end collecting the registration video frame sequence multiple times to generate multiple candidate registration biometric vectors; Calculate the Euclidean distance between the multiple candidate registered biometric vectors, and select the candidate registered biometric vector corresponding to the center point of the Euclidean distance as the final registered biometric vector; The final registered biometric vector is encrypted using the public key of the receiving end and then sent. After the receiving end decrypts and obtains the final registered biometric vector, it stores the final registered biometric vector through a secure multi-party computation protocol.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention uses the spatiotemporal frequency domain features of micro-arterial pulsation extracted from a live face video stream as both the basis for liveness detection and the source of the encryption key. The first branch of a dual-branch deep network outputs the liveness detection result, while the second branch maps the spatiotemporal frequency domain response spectrum into a biometric vector and quantizes it into a dynamic symmetric key for encrypting the data to be transmitted, making the encryption key physiologically dependent on the liveness detection. When the liveness detection result indicates a non-liveness status, the second branch outputs a random perturbation vector to replace the dynamic symmetric key, generating interference ciphertext to block the encryption link. This overcomes the defect of the encryption key being disconnected from the identity, blocking the path to decrypt or forge encrypted data using an intercepted key in a non-liveness state.

[0016] 2. By constructing a facial topology map and performing temporal frequency domain transformation along the time dimension to extract pulse wave frequency band signals, and combining this with spatial dimension graph convolution aggregation to generate a spatiotemporal frequency domain response map, the subtle color change physical features of the face with the heartbeat cycle were extracted. Locality-sensitive hashing (LSH) and key derivation functions were used to convert floating-point biometric vectors into dynamic symmetric keys, achieving stable feature-to-key quantization. Weighted fusion was performed based on temporal cosine similarity and absolute difference features of joint features within a continuous time window, adding temporal dimension liveness detection constraints. During the registration phase, candidate registration biometric vectors corresponding to the Euclidean distance center point were selected and stored using a secure multi-party computation protocol, reducing biometric fluctuation interference and improving the security level of registration template storage. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the overall implementation of the deep learning-based live face recognition and encrypted transmission method of the present invention. Figure 2 This is a flowchart illustrating the implementation of the spatiotemporal graph convolutional network for extracting facial spatiotemporal frequency domain response maps according to the present invention. Figure 3 This is a flowchart illustrating the implementation of the liveness detection result output of the first branch of the dual-branch deep network of the present invention. Figure 4 This is a flowchart illustrating the implementation of dynamic symmetric key generation for the second branch of the dual-branch deep network according to the present invention. Figure 5 This is a flowchart illustrating the implementation of encrypted data decryption and legality verification at the receiving end of the present invention. Figure 6 This is a flowchart illustrating the implementation of the encrypted transmission link blocking method in a non-living state according to the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Please refer to Figure 1 This embodiment provides a deep learning-based method for live face recognition and encrypted transmission. It acquires a continuous sequence of face video frames and constructs a spatiotemporal graph convolutional network to extract subtle color change features of facial region pixels in the video frame sequence as the heartbeat cycle occurs, obtaining a spatiotemporal frequency domain response map. The spatiotemporal frequency domain response map is input into a dual-branch deep network. The first branch of the dual-branch deep network classifies the spatiotemporal frequency domain response map and outputs a liveness detection result. Simultaneously, the second branch of the dual-branch deep network maps the spatiotemporal frequency domain response map into a fixed-length biometric vector, and quantizes the biometric vector into a dynamic symmetric key. During the encrypted transmission stage, the dynamic symmetric key is used to encrypt the recognition result and business data to be transmitted, generating ciphertext data which is sent to the receiving end. The receiving end decrypts the ciphertext data using the pre-securely exchanged biometric vector. When the liveness detection result is non-live, the second branch of the dual-branch deep network outputs a random perturbation vector to replace the dynamic symmetric key, blocking the encrypted link.

[0020] Specifically, refer to Figure 2 When acquiring a continuous sequence of face video frames, face videos of duration T are acquired at a fixed frame rate, resulting in a video frame sequence F containing N frames. ,in This represents the face image in frame t. H represents the image height, W represents the image width, and 3 represents the number of RGB color channels. Face detection and alignment are performed on each frame of the face image, extracting the facial region and normalizing it to a fixed size. When constructing the spatiotemporal graph convolutional network, facial landmark detection is first performed on each frame of the face image, obtaining the coordinates of 68 facial landmarks. ,in This represents the pixel coordinates of the i-th keypoint in frame t. A facial topology map structure is constructed based on the detected facial landmarks. , where the set of nodes Corresponding to 68 facial spatial locations, edge set This corresponds to the spatial adjacency relationship of the face. For any two nodes... and If they are directly adjacent in facial anatomy, then in Add an undirected edge .

[0021] Facial topology structure of N consecutive frames Input a spatiotemporal graph convolutional network. The spatiotemporal graph convolutional network includes temporal dimension processing modules and spatial dimension processing modules. The temporal dimension processing module performs a temporal-frequency domain transformation on the pixel values ​​of the same node in different face video frames along the temporal dimension, extracting the pulse wave frequency band signal corresponding to the node. For each node... Its RGB pixel value sequence in N frames is ,in This represents the RGB pixel value at the i-th node position in frame t. Perform a Fast Fourier Transform on each color channel to obtain the spectral distribution. Where f is the frequency. The target frequency band corresponding to the cardiac cycle is located in the spectral distribution, typically from 0.8Hz to 2.5Hz, and the original signal within this band is extracted as the initial pulse wave signal. An adaptive Wiener filter is used to perform frequency domain noise reduction on the initial pulse wave signal, and the signal whose signal-to-noise ratio meets the preset condition after noise reduction is determined as the pulse wave frequency band signal. .

[0022] The spatial dimension processing module performs graph convolution aggregation operations on the pulse wave frequency band signals along the edges of the facial topology graph structure. ; in, Let l be the node feature matrix of the l-th layer. Let l be the feature dimension of the l-th layer; To add self-loops, the adjacency matrix is ​​A, where A is the adjacency matrix of the face topology graph and I is the identity matrix. for The degree matrix satisfies ; Let L be the learnable weight matrix of the l-th layer; The ReLU function is used as the non-linear activation function. After multi-layer graph convolution aggregation, the frequency domain features after spatial dimension aggregation are obtained. Where d is the final feature dimension. The aggregated frequency domain features are then subjected to spectral reconstruction to generate a spatiotemporal frequency domain response map. ,in For the height of the map, For the width of the map, This represents the number of channels.

[0023] The spatiotemporal frequency domain response map M is input into a two-branch deep network. The two-branch deep network includes a shared feature extraction layer, a first-branch classifier, and a second-branch feature mapping layer. The shared feature extraction layer consists of multiple convolutional and pooling layers, used to extract general features from the spatiotemporal frequency domain response map. The output feature map of the shared feature extraction layer is simultaneously input into both the first and second branches. The output feature map of the shared feature extraction layer is represented as follows: ,in The height of the feature map, The width of the feature map. This represents the number of channels in the feature map.

[0024] refer to Figure 3 The first branch classifier classifies the spatiotemporal frequency domain response map and outputs the liveness detection result. The first branch classifier contains multiple convolutional layers, pooling layers, and fully connected layers. It shares feature maps. Input the first branch and extract frequency distribution features. and energy concentration characteristics The frequency distribution features and energy concentration features are concatenated to obtain joint features. The joint features are input into a probability discriminant function based on a Gaussian mixture model to calculate the posterior probability that the joint features belong to the distribution of real in vivo physiological signals. A Gaussian mixture model consists of K Gaussian components, and its probability density function is: ; in, The weight of the k-th Gaussian component satisfies ; Let be the mean vector of the k-th Gaussian component; Let be the covariance matrix of the k-th Gaussian component. The posterior probability is calculated as follows: ; in, This represents the conditional probability density of real living samples. This represents the conditional probability density of non-living samples. and These are the prior probabilities of a living person and a non-living person, respectively. When the posterior probability exceeds a preset probability threshold... When the posterior probability is less than or equal to a preset probability threshold, the output indicates that the organism is alive; when the posterior probability is less than or equal to a preset probability threshold, the output indicates that the organism is alive. At that time, the output represents the liveness determination result of the non-live organism, with a preset probability threshold. The threshold that optimizes the accuracy of liveness detection is selected by using a grid search method on the validation set. The value ranges from 0.5 to 0.9, and is preferably 0.75.

[0025] refer to Figure 4The second branch maps the spatiotemporal frequency domain response map into a fixed-length biometric vector and quantizes the biometric vector into a dynamic symmetric key. The second branch contains multiple convolutional layers, pooling layers, and multiple fully connected layers. It shares the feature map... Flattened into a one-dimensional vector, it is sequentially input into three fully connected layers. The first layer outputs a dimension of 1024, the second layer outputs a dimension of 512, and the third layer outputs a dimension of 256. The Tanh activation function is used to constrain the output values ​​to the range [-1, 1], resulting in a floating-point biometric feature vector. Perform locality-sensitive hashing on the floating-point biometric vector to convert it into a fixed-length binary bit string. Where L is the length of the bit string. A dynamic symmetric key is generated based on the binary bit string using the SM3 key derivation function. The dynamic symmetric key is dynamically updated based on the temporal evolution of the spatiotemporal frequency domain response map, with the update period synchronized with the video frame acquisition period.

[0026] During the encrypted transmission phase, a dynamic symmetric key k is used to detect the identification result to be transmitted. and business data Encryption is performed. The SM4 symmetric encryption algorithm (China's national standard cryptographic standard) is used to perform the encryption operation, generating ciphertext data. ,in This indicates a data concatenation operation. The encrypted data C is sent to the receiving end. The receiving end decrypts the encrypted data using pre-securely exchanged biometric vectors. During the initial registration phase, the sending end collects a sequence of registration video frames and generates a registration biometric vector. The registered biometric vector is encrypted using an asymmetric encryption algorithm and then sent to the receiving end. During the encrypted transmission phase, the receiving end uses its private key to decrypt and obtain the registered biometric vector. Generate a registration dynamic symmetric key based on the registered biometric vector. The receiving end receives the encrypted data C and uses the registered dynamic symmetric key. The encrypted data is decrypted and verified. If decryption is successful, the recognition result is confirmed to be valid. If decryption fails, the encrypted data is discarded.

[0027] When the liveness detection result is negative, the perturbation generation mechanism of the second branch of the dual-branch deep network is triggered. This is based on the current system timestamp. With a hardware random number generator, generate random perturbation vectors. The dynamic symmetric key is replaced with a random perturbation vector. This random perturbation vector is then used to encrypt the identification result and business data to be transmitted, generating an undecryptable interference ciphertext. ,in The random key is generated by quantizing a random perturbation vector. The interference ciphertext is sent to the receiver, causing the receiver to fail to decrypt based on the pre-securely exchanged biometric vector, thus blocking the encrypted link. The parameters of the dual-branch deep network structure are shown in Table 1.

[0028] Table 1. Parameters of Dual-Branch Deep Network Structure:

[0029] Table 1 shows the structural parameters of the dual-branch deep network. The shared feature extraction layer contains three convolutional layers, progressively decreasing the feature map size and increasing the number of channels. The first branch classifier contains one convolutional layer and two fully connected layers, ultimately outputting a 2D classification result, corresponding to the probabilities of live and non-live individuals. The second branch feature mapping layer contains one convolutional layer and two fully connected layers, ultimately outputting a 256-dimensional floating-point biometric feature vector. The kernel size, stride, and activation function of each layer are designed according to task requirements to balance feature extraction capability and computational efficiency.

[0030] In this embodiment, the spatiotemporal frequency domain features of microarterial pulsation extracted from the live face video stream are used simultaneously as the basis for liveness detection and the source of the encryption key, making the encryption key physiologically dependent on the liveness detection. When the liveness detection result is non-live, a random perturbation vector is output to replace the dynamic symmetric key to generate interference ciphertext to block the encryption link. This overcomes the defect of the encryption key being disconnected from the identity and blocks the path to decrypt or forge encrypted data using the intercepted key in a non-live state.

[0031] In a preferred embodiment, facial landmark detection is performed on a sequence of facial video frames. A facial topology graph is constructed based on the detected landmarks, where nodes in the graph correspond to spatial locations of the face, and edges correspond to spatial adjacency relationships. The facial topology graph is input into a spatiotemporal graph convolutional network. A temporal frequency domain transformation is performed on the pixel values ​​of the same node in different facial video frame sequences along the time dimension to extract the pulse wave frequency band signal corresponding to the node. A graph convolution aggregation operation is then performed on the edges of the facial topology graph along the spatial dimension to aggregate the pulse wave frequency band signal. The aggregated frequency domain features are then subjected to spectral reconstruction to generate a spatiotemporal frequency domain response map.

[0032] Specifically, facial landmark detection employs a convolutional neural network-based landmark detection model. This model takes a face image as input and outputs the coordinates of 68 facial landmarks. These 68 landmarks are distributed across different regions of the face, including the contours, eyebrows, eyes, nose, and mouth. The regional distribution of these 68 facial landmarks is shown in Table 2.

[0033] Table 2. Distribution of Facial Key Points:

[0034] Table 2 divides the 68 facial key points into 10 regions, each corresponding to a different anatomical structure of the face. When constructing the facial topology based on these key points, the connection relationships of the edges are determined according to the adjacency of the facial anatomical structures. For example, the 17 key points of the facial contour are connected sequentially to form the outer contour of the face; the 5 key points of the left eyebrow are connected sequentially to form the contour of the left eyebrow; and the 6 key points of the left eye are connected sequentially to form the contour of the left eye. At the same time, connections are also established between key points in adjacent regions, such as the key points of the bridge of the nose connecting to the key points of the tip of the nose, and the key points of the eyes connecting to the key points of the eyebrows.

[0035] A temporal-frequency domain transform is performed on the pixel values ​​of the same node in the facial topology structure across different face video frame sequences along the time dimension to extract the pulse wave frequency band signal corresponding to the node. For each node... Its RGB pixel value sequence in N frames is ,in This represents the RGB pixel value at the i-th node position in frame t. Perform a Fast Fourier Transform on each color channel to obtain the spectral distribution. Where c represents the color channel, c∈{R,G,B}. The formula for calculating the Fast Fourier Transform is: ; in, k is the total number of frames in the video frame sequence, and k is the frequency index. , Let be the pixel value of the c-th color channel at the i-th node position in frame t. The corresponding frequency is . ,in This refers to the video sampling frame rate.

[0036] The target frequency band corresponding to the cardiac cycle is located in the spectral distribution, typically from 0.8 Hz to 2.5 Hz, and the original signal within this frequency band is extracted as the initial pulse wave signal. An adaptive Wiener filter is used to perform frequency domain noise reduction on the initial pulse wave signal. The frequency response of the adaptive Wiener filter is: ; in, For the signal power spectral density, This refers to the noise power spectral density. The power is estimated by averaging the power of components outside the target frequency band (0.8Hz to 2.5Hz), specifically as follows: ; in, Indicates color channels; This is a frequency variable, with the unit being Hz; For the first Each node in the color channel Spectral distribution on; Indicates the number of frequency points outside the frequency band; To estimate the obtained noise power spectral density.

[0037] The denoised pulse wave signal is as follows: ;

[0038] Calculate the signal-to-noise ratio of the denoised signal: ; When the signal-to-noise ratio is greater than the preset signal-to-noise ratio threshold At that time, the signal of that color channel is determined as the effective pulse wave frequency band signal, and a preset signal-to-noise ratio threshold is set. The signal-to-noise ratio (SNR) of the effective pulse wave signal was determined through experimental statistical analysis, with a range of 5 dB to 15 dB, preferably 10 dB. The effective pulse wave frequency band signals from the three color channels were then weighted and fused to obtain the final pulse wave frequency band signal. : ; in, The weights of each color channel satisfy the following conditions: The weights are adaptively adjusted based on the signal-to-noise ratio (SNR) of each color channel, with channels having higher SNRs receiving larger weights.

[0039] Graph convolution aggregation is performed on the edges of the facial topology graph along the spatial dimension of the pulse wave frequency band signal. An adjacency matrix of the facial topology graph is then constructed. ,in Represents a node and There are edges between them. This indicates that no edge exists. Adding a self-loop yields the result. , where I is the identity matrix. Calculate the degree matrix. ,in .

[0040] The spatiotemporal graph convolutional network contains L graph convolutional layers. The input to the l-th graph convolutional layer is the node feature matrix. ,in The initial node features are the pulse wave frequency band signals corresponding to each node. The output of the l-th graph convolutional layer is: ; in, Let be the learnable weight matrix of the l-th layer. The ReLU activation function is used. After L layers of graph convolution aggregation, the frequency domain features after spatial dimension aggregation are obtained. .

[0041] The aggregated frequency domain features are then subjected to spectral reconstruction to generate a spatiotemporal frequency domain response map. First, the node feature matrix is... The facial key points are arranged according to their spatial positions to obtain a two-dimensional feature map. Then, an upsampling operation is performed on the two-dimensional feature map to adjust its size. Finally, channel dimensions are added to generate a spatiotemporal frequency domain response map. ,in This represents the number of channels.

[0042] The spatiotemporal frequency domain response map is input into the first branch classifier to extract frequency distribution features and energy concentration features. A feature concatenation operation is performed on the frequency distribution features and energy concentration features. The concatenated joint features are then input into a probability discriminant function based on a Gaussian mixture model to calculate the posterior probability that the joint features belong to the distribution of real living physiological signals. When the posterior probability is greater than a preset probability threshold, a liveness detection result representing a living organism is output; when the posterior probability is less than or equal to the preset probability threshold, a liveness detection result representing a non-living organism is output.

[0043] Furthermore, multiple joint features corresponding to multiple face video frame sequences within a continuous time window are obtained. The temporal cosine similarity between the joint features of adjacent time slices is calculated, and the absolute difference feature of the joint features of adjacent time slices is extracted. A weighted fusion operation is performed on the posterior probability, temporal cosine similarity, and absolute difference feature to obtain a comprehensive liveness confidence score. When the comprehensive liveness confidence score is greater than a preset comprehensive threshold, a liveness determination result representing a live subject is output; when the comprehensive liveness confidence score is less than or equal to the preset comprehensive threshold, a liveness determination result representing a non-live subject is output.

[0044] Specifically, the time window length is set to It contains K consecutive time slices. Each time slice corresponds to a sequence of face video frames, generating a joint feature. Where k = 1, 2, ..., K. Calculate the temporal cosine similarity between the joint features of adjacent time slices: ; in, Represents the vector dot product. Represents the L2 norm of a vector. Extract the absolute difference feature of the joint features of adjacent time slices: .

[0045] Calculate the mean of the absolute difference characteristic:

[0046] in, Let i be the i-th element of the absolute value feature of the difference.

[0047] posterior probability Temporal cosine similarity and the mean of the absolute value of the difference characteristics Perform a weighted fusion operation to obtain the overall liveness confidence score: ; in, , , For the weighting coefficients, satisfying The weighting coefficients are optimized and determined based on the training data. The average of all comprehensive liveness confidence scores within the time window is calculated. ; When the average comprehensive liveness confidence level Greater than the preset comprehensive threshold When the average comprehensive liveness confidence score is 1, the output is the liveness determination result representing the liveness of the organism. Less than or equal to the preset comprehensive threshold At that time, the output represents the liveness determination result of the non-live organism, with a preset comprehensive threshold. The threshold that minimizes the equal error rate (EER) is determined by ROC curve analysis on an independent test set, with a value ranging from 0.6 to 0.9, preferably 0.75.

[0048] In this embodiment, a facial topology map is constructed, and a temporal frequency domain transformation is performed along the time dimension to extract the pulse wave frequency band signal. This is combined with spatial dimension graph convolution aggregation to generate a spatiotemporal frequency domain response map, extracting the subtle color changes in the face during the cardiac cycle. Weighted fusion is performed based on temporal cosine similarity and absolute difference features of joint features within a continuous time window, adding temporal dimension liveness detection constraints and improving the accuracy of liveness detection.

[0049] In another preferred embodiment, the spatiotemporal frequency domain response map is input into a multilayer fully connected layer of the second branch, and the spatiotemporal frequency domain response map is dimensionality-reduced and mapped into a fixed-length floating-point biometric vector. Locality-sensitive hashing is performed on the floating-point biometric vector to convert it into a fixed-length binary bit string. Based on the binary bit string, a dynamic symmetric key is generated using a national cryptographic key derivation function, and the dynamic symmetric key is dynamically updated according to the temporal evolution of the spatiotemporal frequency domain response map.

[0050] Specifically, the second branch consists of three fully connected layers. These layers share feature maps. Flattened into a one-dimensional vector The flattened vector is input into the first fully connected layer, which outputs a feature vector with a dimension of 1024. The output of the first fully connected layer is input into the second fully connected layer, which outputs a feature vector with a dimension of 512. The output of the second fully connected layer is input into the third fully connected layer, which outputs a floating-point biometric feature vector with a dimension of 256. The third fully connected layer uses the Tanh activation function to restrict each element of the biofeature vector to the range [-1, 1].

[0051] Locality-Sensitive Hash (LSH) is performed on floating-point biometric vectors to convert them into fixed-length binary bit strings. The LSH operation employs a random hyperplane hashing method. M clusters of random hyperplane hash functions are generated, each containing K random hyperplanes. The parameter settings for the LSH functions are shown in Table 3.

[0052] Table 3. Parameters of Locality Sensitive Hash Functions:

[0053] In Table 3, there are 16 locality-sensitive hash function clusters, each containing 8 hash functions, with a total bit string length of 128 bits. Each random hyperplane is composed of a random vector. and a bias term Define a random vector, where m = 1, 2, ..., M and k = 1, 2, ..., K. Each element follows a standard normal distribution, and the bias term... It follows a uniform distribution on [0,1].

[0054] For floating-point biological feature vectors The output of the k-th hash function in the m-th group is: ; in, This represents the vector dot product. The m-th hash function generates a local binary substring of length K. Concatenate M local binary substrings in a predetermined order to form a binary bit string of total length L = M × K. .

[0055] A dynamic symmetric key is generated by inputting a binary bit string into a key derivation function based on a hash message authentication code and adding the current timestamp salt value. The key derivation function uses the Chinese national cryptographic algorithm SM3. Current timestamp salt value. Precise to the second, ensuring a different key is generated every second. The dynamic symmetric key generation process is as follows: ; in, This indicates a data concatenation operation. The SM3 algorithm outputs a 256-bit hash value, and the first 128 bits are used as the dynamic symmetric key. The dynamic symmetric key is updated once per second, synchronized with the video frame acquisition cycle.

[0056] refer to Figure 5 During the initial registration phase, the sending end collects a sequence of registration video frames and generates a registration biometric vector. This biometric vector is then encrypted using an asymmetric encryption algorithm and sent to the receiving end. In the encrypted transmission phase, the receiving end uses its private key to decrypt and obtain the registration biometric vector, and generates a dynamic symmetric key based on this vector. The receiving end receives the encrypted data and uses the dynamic symmetric key to decrypt and verify it. If decryption is successful, the identification result is confirmed as legitimate; if decryption fails, the encrypted data is discarded.

[0057] Specifically, during the initial registration phase, the sending end repeatedly collects registration video frame sequences to generate multiple candidate registration biometric vectors. The number of collections is [number missing]. The frequency is typically set to 5 to 10 times. Each data collection session lasts for [duration missing]. Generate a candidate registration biometric vector ,in Calculate the Euclidean distance matrix between multiple candidate registered biometric feature vectors. ,in Let be the Euclidean distance between the i-th and j-th candidate registration biometric vectors. Calculate the average Euclidean distance between each candidate registration biometric vector and all other candidate registration biometric vectors: ; The candidate registered biometric vector with the smallest average Euclidean distance is selected as the final registered biometric vector. ,Right now: ; The final registered biometric vector is the center point of the Euclidean distance of all candidate registered biometric vectors, which can reduce the interference of biometric fluctuations.

[0058] Using the receiver's public key The final registered biometric vector is encrypted and sent. The asymmetric encryption algorithm used is the Chinese national standard SM2 algorithm. The encryption process is as follows: ;

[0059] The encrypted registration biometric vector Send to the receiving end. The receiving end uses its own private key. Decryption to obtain the final registered biometric vector: ; After the receiving end decrypts and obtains the final registered biometric vector, it stores the final registered biometric vector using a secure multi-party computation protocol. This protocol divides the registered biometric vector into multiple shares, each stored on a separate server. Only when multiple servers participate in the computation can the complete registered biometric vector be recovered, thus enhancing the security level of the registered template storage.

[0060] During the encrypted transmission phase, the receiving end generates a registered dynamic symmetric key based on the registered biometric vector. The generation process is the same as that of the sending end's dynamic symmetric key generation process. The receiving end obtains the current timestamp. The same locality-sensitive hash operation is performed on the registered biometric vectors to generate binary bit strings. Then, a registration dynamic symmetric key is generated using the national cryptographic SM3 algorithm: ; The receiving end receives the encrypted data C and uses the registered dynamic symmetric key. Decrypt and verify the encrypted data: ; in, This indicates the SM4 decryption algorithm. If decryption is successful, a valid recognition result R and business data D are obtained, and the recognition result is confirmed to be legitimate; if decryption fails, garbled data is obtained, and the ciphertext data is discarded.

[0061] refer to Figure 6 When the liveness detection result is not live, the perturbation generation mechanism of the second branch of the dual-branch deep network is triggered. Based on the current system timestamp and a hardware random number generator, a random perturbation vector is generated. This random perturbation vector replaces the dynamic symmetric key, and the identification result and business data to be transmitted are encrypted using this vector. An undecryptable interference ciphertext is generated and sent to the receiving end, causing the receiving end to fail to decrypt based on the pre-securely exchanged biometric vector, thus blocking the encrypted link.

[0062] Specifically, when the liveness detection result is non-liveness, a random perturbation vector is generated. Each element of the random perturbation vector follows a standard normal distribution. To increase randomness, the current system timestamp is used. The hardware random number generator serves as the seed input. It generates truly random numbers based on physical noise sources, ensuring the unpredictability of the random perturbation vector.

[0063] random perturbation vector The input follows the same quantization process as the biometric vector to generate a random key. Perform a locality-sensitive hash operation on the random perturbation vector to generate a random binary bit string. Then, a random key is generated using the Chinese national cryptographic algorithm SM3: ; Using random keys Encrypt the identification result R and the business data D to be transmitted, generating interference ciphertext: ; Interference ciphertext The data is sent to the receiving end. Since the random perturbation vector is independent of the pre-securely exchanged registration biometric vector, the receiving end generates a dynamic symmetric key based on the registration biometric vector. Unable to decrypt the jammed ciphertext, thus blocking the encrypted link.

[0064] In this embodiment, locality-sensitive hashing (LSH) and key derivation functions are used to convert floating-point biometric vectors into dynamic symmetric keys, achieving stable quantization from features to keys. During the registration phase, candidate biometric vectors corresponding to the center point of Euclidean distance are selected and stored using a secure multi-party computation protocol, reducing interference from biometric fluctuations and improving the security level of registration template storage. When the liveness detection result is negative, a random perturbation vector is output to replace the dynamic symmetric key, generating interference ciphertext, effectively blocking encrypted transmission in the negative state.

Claims

1. A method for live face recognition and encrypted transmission based on deep learning, characterized in that, include: A continuous sequence of face video frames is acquired, a spatiotemporal graph convolutional network is constructed, and a temporal frequency domain transformation is performed on the pixel values ​​of the same node in different video frames along the time dimension to extract the pulse wave frequency band signal. A graph convolution aggregation operation is performed on the pulse wave frequency band signal along the spatial dimension to generate a spatiotemporal frequency domain response map. The spatiotemporal frequency domain response map is input into a dual-branch deep network, and the first branch classifier of the dual-branch deep network is used to classify the spatiotemporal frequency domain response map and output the liveness detection result. Simultaneously, the spatiotemporal frequency domain response spectrum is input into the multilayer fully connected layer through the second branch of the dual-branch deep network and dimensionality reduction mapping is transformed into a fixed-length floating-point biometric vector. Then, local sensitive hashing is performed on the floating-point biometric vector to convert it into a binary bit string. Finally, a dynamic symmetric key is generated through the key derivation function. During the encrypted transmission phase, the identification result and business data to be transmitted are encrypted using the dynamic symmetric key to generate ciphertext data and send it to the receiving end. The receiving end decrypts the ciphertext data using the biometric vector that has been securely exchanged in advance. When the liveness detection result is non-liveness, the second branch of the dual-branch deep network outputs a random perturbation vector to replace the dynamic symmetric key, thereby blocking the encrypted link.

2. The method for live face recognition and encrypted transmission based on deep learning according to claim 1, characterized in that, The step of constructing a spatiotemporal graph convolutional network to extract the subtle color change features of facial region pixels in the face video frame sequence as the heartbeat cycle generates, and obtaining a spatiotemporal frequency domain response map, includes: performing facial key point detection on the face video frame sequence, and constructing a facial topology graph structure based on the detected facial key points, wherein the nodes of the facial topology graph structure correspond to the facial spatial location, and the edges of the facial topology graph structure correspond to the facial spatial adjacency relationship; The facial topology graph structure is input into the spatiotemporal graph convolutional network. The pixel values ​​of the same node in the facial topology graph structure in different facial video frame sequences are transformed along the time dimension to extract the pulse wave frequency band signal corresponding to the node. A graph convolution aggregation operation is performed on the pulse wave frequency band signal along the spatial dimension on the edges of the facial topology structure. The aggregated frequency domain features are then subjected to spectral reconstruction to generate the spatiotemporal frequency domain response map.

3. The method for live face recognition and encrypted transmission based on deep learning according to claim 1, characterized in that, The step of classifying the spatiotemporal frequency domain response map and outputting the liveness detection result by the first branch classifier of the dual-branch deep network includes: inputting the spatiotemporal frequency domain response map into the first branch classifier and extracting the frequency distribution features and energy concentration features in the spatiotemporal frequency domain response map. The frequency distribution feature and the energy concentration feature are concatenated, and the concatenated joint feature is input into a probability discriminant function based on a Gaussian mixture model to calculate the posterior probability that the joint feature belongs to the distribution of real living physiological signals. When the posterior probability is greater than a preset probability threshold, the liveness determination result representing a live body is output; when the posterior probability is less than or equal to the preset probability threshold, the liveness determination result representing a non-live body is output.

4. The method for live face recognition and encrypted transmission based on deep learning according to claim 1, characterized in that, The steps of mapping the spatiotemporal frequency domain response map into a fixed-length biometric vector through the second branch of the dual-branch deep network and quantizing the biometric vector into a dynamic symmetric key include: inputting the spatiotemporal frequency domain response map into a multilayer fully connected layer of the second branch, and dimensionality-reducing the spatiotemporal frequency domain response map into the fixed-length floating-point biometric vector; Perform a locality-sensitive hash operation on the floating-point biometric vector to convert the floating-point biometric vector into a fixed-length binary bit string; Based on the binary bit string, the dynamic symmetric key is generated by calculating the national cryptographic key derivation function, and the dynamic symmetric key is dynamically updated as the spatiotemporal frequency domain response spectrum evolves over time.

5. The method for live face recognition and encrypted transmission based on deep learning according to claim 1, characterized in that, The step of the receiving end decrypting the encrypted data using the pre-securely exchanged biometric vector includes: in the initial registration stage, the sending end collects the registration video frame sequence and generates the registration biometric vector, and encrypts the registration biometric vector using an asymmetric encryption algorithm before sending it to the receiving end; During the encrypted transmission phase, the receiving end uses a private key to decrypt and obtain the registered biometric vector, and generates a registration dynamic symmetric key based on the registered biometric vector; The receiving end receives the ciphertext data and uses the registered dynamic symmetric key to decrypt and verify the ciphertext data. If the decryption is successful, the identification result is confirmed to be valid; if the decryption fails, the ciphertext data is discarded.

6. The method for live face recognition and encrypted transmission based on deep learning according to claim 1, characterized in that, When the liveness detection result is non-live, the step of the second branch of the dual-branch deep network outputting a random perturbation vector to replace the dynamic symmetric key and block the encrypted link includes: when the liveness detection result is non-live, triggering the perturbation generation mechanism of the second branch of the dual-branch deep network; The random perturbation vector is generated based on the current system timestamp and the hardware random number generator; The dynamic symmetric key is replaced by the random perturbation vector. The identification result and the business data to be transmitted are encrypted using the random perturbation vector to generate an undecryptable interference ciphertext, which is then sent to the receiving end. This causes the receiving end to fail to decrypt the biometric vector based on the pre-securely exchanged biometric vector, thereby blocking the encrypted link.

7. The method for live face recognition and encrypted transmission based on deep learning according to claim 2, characterized in that, The step of performing a time-series frequency domain transformation on the pixel values ​​of the same node in the facial topology structure in different facial video frame sequences along the time dimension to extract the pulse wave frequency band signal corresponding to the node includes: performing a fast Fourier transform on the pixel values ​​of the same node along the time dimension to obtain the spectral distribution of the same node; Locate the target frequency band corresponding to the cardiac cycle in the spectrum distribution, and extract the original signal in the target frequency band as the initial pulse wave signal; The initial pulse wave signal is subjected to frequency domain noise reduction using an adaptive Wiener filter, and the signal whose signal-to-noise ratio meets the preset condition after noise reduction is determined as the pulse wave frequency band signal.

8. The method for live face recognition and encrypted transmission based on deep learning according to claim 3, characterized in that, After calculating the posterior probability that the joint features belong to the distribution of real living physiological signals, the method further includes: obtaining multiple joint features corresponding to multiple face video frame sequences within a continuous time window; Calculate the temporal cosine similarity between the joint features of adjacent time slices, and extract the difference absolute value features of the joint features of adjacent time slices; The posterior probability, the temporal cosine similarity, and the absolute difference feature are weighted and fused to obtain the comprehensive liveness confidence. When the overall liveness confidence score is greater than a preset overall threshold, the liveness determination result representing a live body is output; when the overall liveness confidence score is less than or equal to the preset overall threshold, the liveness determination result representing a non-live body is output.

9. The method for live face recognition and encrypted transmission based on deep learning according to claim 4, characterized in that, The step of performing locality-sensitive hashing on the floating-point biometric vector to convert the floating-point biometric vector into a fixed-length binary bit string includes: generating multiple sets of random hyperplane hash function clusters; The floating-point biometric vector is subjected to a fragmented hash mapping using the multiple sets of random hyperplane hash function clusters to obtain multiple local binary substrings. The multiple local binary substrings are concatenated into the binary bit string in a preset order; The binary bit string is input into a key derivation function based on a hash message authentication code, and the current timestamp salt value is added to generate the dynamic symmetric key.

10. A method for live face recognition and encrypted transmission based on deep learning according to claim 5, characterized in that, In the initial registration phase, the sending end collects the registration video frame sequence and generates a registration biometric vector, and encrypts the registration biometric vector using an asymmetric encryption algorithm before sending it to the receiving end. The steps include: in the initial registration phase, the sending end collects the registration video frame sequence multiple times and generates multiple candidate registration biometric vectors. Calculate the Euclidean distance between the multiple candidate registered biometric vectors, and select the candidate registered biometric vector corresponding to the center point of the Euclidean distance as the final registered biometric vector; The final registered biometric vector is encrypted using the public key of the receiving end and then sent. After the receiving end decrypts and obtains the final registered biometric vector, it stores the final registered biometric vector through a secure multi-party computation protocol.

Images