A method for detecting live human faces based on a near-infrared and visible light binocular camera

Abstract

The present invention discloses a live face detection method based on a binocular camera of near-infrared and visible light, belonging to the technical field of face recognition. The method includes collecting binocular visible light and near-infrared images and performing cross-modal alignment; extracting the subcutaneous microvascular pulsation time series signal in the near-infrared image, combining the near-infrared image sequence to enhance the texture boundary features in the visible light image sequence, and obtaining an enhanced texture feature map; analyzing the 3D deformation continuity and physiological motion consistency in the expression actions based on the enhanced texture feature map; and combining live detection through time-frequency domain perturbation detection. The present invention adopts multi-modal time series synchronous material reflection gradient analysis and microvascular pulsation time series tracking technology, combines the biomechanical model constraint of facial dynamic 3D deformation and time-frequency joint anti-interference verification, and can effectively improve the accuracy and robustness of live detection, providing a strong guarantee for the security of face recognition technology.

Description

Technical Field

[0001] The present invention relates to the field of face recognition technology, and in particular to a method for detecting a live human face based on a binocular camera of near-infrared and visible light. Background Art

[0002] With the wide application of face recognition technology, the live detection technology has become a key link to ensure the security of face recognition. Traditional live detection methods mainly rely on a single image modality, such as visible light images. These methods are vulnerable to lighting conditions, image quality, and forgery means, such as photos, videos, or 3D printed masks, resulting in insufficient accuracy and robustness of live detection.

[0003] Currently, the existing live detection methods rely on single-modal solutions. Due to physiological feature blind spots and material reflection noise interference, the false negative rate of high-fidelity forgery attacks is relatively high; the spatio-temporal coordination ability of multi-modal data is weak, and cross-modal image alignment depends on manual calibration rather than dynamic feature registration, weakening the joint discrimination effect of silicone abnormal reflection and natural skin texture; dynamic deformation detection lacks biomechanical constraints, and single-eye optical flow tracking cannot distinguish between real skin elastic deformation and the mechanical motion mode of forgeries, making it difficult to resist dynamic forgery attacks; time-frequency domain analysis methods have insufficient sensitivity to the high-frequency singularity of materials, and poor robustness against noise attacks, and frequency domain fingerprint matching is prone to failure. Summary of the Invention

[0004] To solve the above problems, the present invention provides a method for detecting a live human face based on a binocular camera of near-infrared and visible light. By using a multi-modal time-series synchronous material reflection gradient analysis and microvascular pulsation time-series tracking technology, combined with the biomechanical model constraint of facial dynamic 3D deformation and time-frequency joint anti-interference verification, it can effectively improve the accuracy and robustness of live detection, providing a strong guarantee for the security of face recognition technology.

[0005] The above object can be achieved by the following solutions:

[0006] A method for detecting a live human face based on a binocular camera of near-infrared and visible light, comprising: synchronously collecting binocular visible light and near-infrared images and performing cross-modal alignment to generate a sequence of binocular visible light images and a sequence of near-infrared images corresponding to pixel-level space; extracting the subcutaneous microvascular pulsation time-series signal from the sequence of near-infrared images; combining the pulsation time-series signal and the sequence of near-infrared images to enhance the texture boundary features in the sequence of visible light images, obtaining an enhanced texture feature map; generating a sequence of facial depth maps based on the enhanced texture feature map, and analyzing the 3D deformation continuity and physiological motion consistency in facial expressions and movements to obtain a deformation continuity analysis result; performing a live determination through time-frequency domain perturbation detection according to the pulsation time-series signal, the enhanced texture feature map, and the deformation continuity analysis result.

[0007] Optionally, the synchronously acquiring binocular visible light and near-infrared images and performing cross-modal alignment to generate a binocular visible light image sequence and a near-infrared image sequence with pixel-level spatial correspondence includes: synchronously triggering a near-infrared camera and a binocular visible light camera to obtain a facial near-infrared image and a binocular visible light image at the same moment; performing cross-modal image alignment through SURF feature point matching to generate a binocular visible light image sequence and a near-infrared image sequence with pixel-level spatial correspondence.

[0008] Optionally, the extracting the subcutaneous microvascular pulsation time series signal from the near-infrared image sequence includes: separating the subcutaneous scattering component from the near-infrared image sequence to obtain a first physiological feature map; eliminating the specular reflection noise of the first physiological feature map based on the Retinex algorithm to obtain a second physiological feature map; using an optical flow method to track the microvascular pulsation signal in the facial area of the second physiological feature map and extracting the periodic blood flow change feature to obtain a pulsation time series signal.

[0009] Optionally, the combining the pulsation time series signal and the near-infrared image sequence to enhance the texture boundary feature in the visible light image sequence to obtain an enhanced texture feature map includes: calculating the microvascular pulsation frequency according to the phase difference of the pulsation time series signal and generating a reflectance sampling window aligned with the heartbeat cycle to obtain window parameters; calculating the reflectance difference in the near-infrared band within the time interval defined by the window parameters to generate a material reflectance anomaly map; performing edge detection on the visible light image to generate a gradient magnitude map; performing channel concatenation on the reflectance anomaly map and the gradient magnitude map and inputting them into a spectral-sensitive convolutional layer to generate spatial attention weights; performing high-frequency filtering on the visible light image to obtain an edge texture feature map; multiplying the attention weights and the edge texture feature map pixel by pixel to generate an enhanced texture feature map.

[0010] Optionally, the generating a facial depth map sequence based on the enhanced texture feature map and analyzing the 3D deformation continuity and physiological motion consistency in the expression action to obtain a deformation continuity analysis result includes: calculating an initial depth map based on binocular disparity according to the enhanced texture feature map; registering the initial depth map with the facial key points of the near-infrared image sequence to obtain a high-precision 3D facial displacement field; using a preset biomechanical model to perform constraint filtering on the facial displacement field to eliminate the deformation vectors beyond the physiological limit to obtain a deformation trajectory.

[0011] Optionally, the determination of liveness through time-frequency domain perturbation detection based on the pulsation timing signal, the enhanced texture feature map, and the result of deformation continuity analysis includes: calculating the time-frequency consistency between the pulsation timing signal and the deformation trajectory to obtain the physiological motion matching degree; using the enhanced texture feature map to detect the high-frequency singularity of the silicone artifact to obtain the material abnormality degree; calculating the wavelet domain energy distribution difference between the near-infrared image sequence and the visible light image sequence. If the phase consistency of the high-frequency sub-band is lower than a preset threshold, it is determined that there is an adversarial noise attack and a liveness alarm is issued; if the phase consistency of the high-frequency sub-band is greater than or equal to the preset threshold, then when the physiological motion matching degree is less than a preset first threshold and the material abnormality degree is greater than a preset second threshold, a liveness alarm is triggered.

[0012] Optionally, the calculating the time-frequency consistency between the pulsation timing signal and the deformation trajectory to obtain the physiological motion matching degree includes: performing time alignment and band-pass filtering on the pulsation timing signal and the deformation trajectory, and respectively extracting the main frequency features within the heartbeat frequency band; decomposing the 3D displacement field of the deformation trajectory into low-dimensional motion modes, and performing power spectrum analysis on each mode; within the frequency band near the heartbeat main frequency, calculating the spectral energy ratio of each motion mode and the coherence with the microvascular signal; fusing the results of the energy ratio and coherence to generate the physiological motion matching degree, and when the physiological motion matching degree exceeds a preset threshold, it is determined that the physiological motion is consistent.

[0013] Optionally, the using the enhanced texture feature map to detect the high-frequency singularity of the silicone artifact to obtain the material abnormality degree includes: performing multi-scale wavelet decomposition on the enhanced texture feature map, and extracting the local singularity points of the non-linear response in the high-frequency sub-band; constructing a frequency domain fingerprint template of the silicone material based on the time-frequency distribution characteristics of the singularity points, and calculating the time-frequency ridge line difference degree between the time-frequency distribution characteristics and the frequency domain fingerprint template; statistically analyzing the kurtosis and skewness indexes of the difference degree in the spatial domain to generate the material abnormality degree, where the material abnormality degree characterizes the deviation degree of the reflection characteristics of the artifact material from the biological tissue.

[0014] Optionally, the method further includes: when the material abnormality degree is greater than the second threshold, reducing the reflectivity sampling window, reducing the reflectivity sampling window from to , where , in the formula, is the sensitivity adjustment coefficient, is the material abnormality degree, is the second threshold; performing edge enhancement operation within the reduced window; calculating the reflectivity difference using the reduced window to obtain a new reflection abnormality map.

[0015] Based on the same inventive concept, the present invention also provides a live face detection system based on a binocular camera of near-infrared and visible light. The system further includes: a multimodal module data processing module, configured to synchronously collect binocular visible light and near-infrared images and perform cross-modal alignment to generate a sequence of binocular visible light images and a sequence of near-infrared images corresponding to each other at the pixel level in space; a physiological feature generation module, configured to extract the subcutaneous microvascular pulsation time series signal in the sequence of near-infrared images; a physical feature generation module, configured to combine the pulsation time series signal and the sequence of near-infrared images to enhance the texture boundary features in the sequence of visible light images and obtain an enhanced texture feature map; a dynamic deformation verification module, configured to generate a sequence of facial depth maps based on the enhanced texture feature map and analyze the 3D deformation continuity and physiological movement consistency in the expression actions to obtain a deformation continuity analysis result; and a live body determination module, configured to perform live body determination through time-frequency domain perturbation detection according to the pulsation time series signal, the enhanced texture feature map, and the deformation continuity analysis result.

[0016] Compared with the prior art, the present invention has the following advantages:

[0017] 1. The present invention synchronously collects binocular visible light and near-infrared images and combines the information of these two modalities for live body detection. The near-infrared image is sensitive to physiological features such as subcutaneous microvessels and can capture the pulsation signal unique to a live body; while the visible light image can present the texture details and expression actions of the human face. This multimodal fusion method effectively improves the accuracy of live body detection and reduces the false judgment that may be caused by a single modality;

[0018] 2. The present invention uses SURF feature point matching for cross-modal image alignment to generate a sequence of binocular visible light images and a sequence of near-infrared images corresponding to each other at the pixel level in space. This technology ensures the precise alignment between different modality images, improves the robustness of subsequent feature extraction and analysis, and enables the system to work stably under different lighting conditions and complex backgrounds;

[0019] 3. The present invention not only extracts the subcutaneous microvascular pulsation time series signal in the near-infrared image as a physiological feature, but also combines this signal to enhance the texture boundary features in the visible light image to obtain an enhanced texture feature map. This method of combining physiological features and physical features further enhances the discrimination ability of the system, enabling it to more accurately identify the difference between a real human face and a forged human face;

[0020] 4. When analyzing the 3D deformation continuity and physiological movement consistency in the expression actions, the present invention uses a preset biomechanical model to perform constrained filtering on the facial displacement field and eliminates the deformation vectors that exceed the physiological limit. This method effectively improves the accuracy of deformation analysis and reduces the false judgment caused by the mechanical movement of forgeries;

[0021] 5. The present invention determines the living body through various means such as calculating the time-frequency consistency between the pulsation time series signal and the deformation trajectory, detecting the high-frequency singularity of the silicone artifact using the enhanced texture feature map, and calculating the difference in the wavelet domain energy distribution of the near-infrared and visible light image sequences. These time-frequency domain perturbation detection methods can effectively resist adversarial noise attacks and other interference factors, improving the anti-interference ability and overall performance of the system.

[0022] Other features and advantages of the present invention will be described in the following specification, and in part will be obvious from the specification, or will be understood by implementing the present invention. The objectives and other advantages of the present invention can be realized and obtained through the structures pointed out in the specification, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or in the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0024] Figure 1 It is a schematic flowchart of a living body face detection method based on a near-infrared and visible light binocular camera according to an embodiment of the present invention.

[0025] Figure 2 It is an execution flowchart of a living body face detection method based on a near-infrared and visible light binocular camera according to an embodiment of the present invention.

[0026] Figure 3 It is a schematic structural diagram of a living body face detection system based on a near-infrared and visible light binocular camera according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0027] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the protection scope of the present invention.

[0028] Refer to Figure 1, an embodiment of the present invention proposes a live face detection method based on a binocular camera of near-infrared and visible light, which adopts a multi-modal time-sequence synchronization material reflection gradient analysis and microvascular pulsation time-sequence tracking technology, combined with the biomechanical model constraint of facial dynamic 3D deformation and time-frequency joint anti-interference verification, and can effectively improve the accuracy and robustness of live detection, providing a strong guarantee for the security of face recognition technology.

[0029] The method of this embodiment specifically includes:

[0030] Synchronously collect binocular visible light and near-infrared images and perform cross-modal alignment to generate a sequence of binocular visible light images and a sequence of near-infrared images corresponding at the pixel level in space;

[0031] Extract the subcutaneous microvascular pulsation time-sequence signal in the sequence of near-infrared images;

[0032] Combine the pulsation time-sequence signal and the sequence of near-infrared images to enhance the texture boundary features in the sequence of visible light images and obtain an enhanced texture feature map;

[0033] Generate a sequence of facial depth maps based on the enhanced texture feature map, and analyze the 3D deformation continuity and physiological movement consistency in the expression actions to obtain a deformation continuity analysis result;

[0034] According to the pulsation time-sequence signal, the enhanced texture feature map, and the deformation continuity analysis result, perform a live determination through time-frequency domain perturbation detection.

[0035] Specifically, by synchronously collecting binocular visible light and near-infrared images and performing cross-modal alignment, the problem that single-modal images are easily affected by lighting conditions, image quality, and forgery means is solved. By extracting the subcutaneous microvascular pulsation time-sequence signal in the sequence of near-infrared images, combining the pulsation time-sequence signal and the sequence of near-infrared images, and enhancing the texture boundary features in the sequence of visible light images, the problems of physiological feature blind spots and material reflection noise interference are solved. By generating a sequence of facial depth maps based on the enhanced texture feature map and analyzing the 3D deformation continuity and physiological movement consistency in the expression actions, the problem that the dynamic deformation detection lacks biomechanical constraints is solved. By performing a live determination through time-frequency domain perturbation detection, the problems of insufficient sensitivity to high-frequency singularities of materials and poor robustness against adversarial noise attacks are solved.

[0036] Exemplarily, such as Figure 2As shown, first, the near-infrared camera and the visible-light binocular camera are synchronously triggered to obtain the facial near-infrared image and the binocular visible-light image at the same moment. Then, SURF feature point matching is used for cross-modal image alignment to generate a sequence of binocular visible-light images and a sequence of near-infrared images that are pixel-level spatially corresponding. On this basis, the subcutaneous scattering components of the near-infrared image sequence are separated to obtain the first physiological feature map; the specular reflection noise of the first physiological feature map is eliminated based on the Retinex algorithm to obtain the second physiological feature map; the optical flow method is used to track the microvascular pulsation signal in the facial area of the second physiological feature map, and the periodic blood flow change characteristics are extracted to obtain the pulsation time series signal. The microvascular pulsation frequency is calculated according to the phase difference of the pulsation time series signal, and a reflectivity sampling window aligned with the heartbeat cycle is generated to obtain window parameters. Within the time interval defined by the window parameters, the reflectivity difference in the near-infrared band is calculated to generate a material reflection anomaly map. Edge detection is performed on the visible-light image to generate a gradient magnitude map; the reflection anomaly map and the gradient magnitude map are channel-cascaded and input into a spectral-sensitive convolutional layer to generate spatial attention weights. High-frequency filtering is performed on the visible-light image to obtain an edge texture feature map; the attention weights and the edge texture feature map are multiplied pixel by pixel to generate an enhanced texture feature map. Based on the enhanced texture feature map, the initial depth map is calculated using binocular disparity; the initial depth map is registered with the facial key points of the near-infrared image sequence to obtain a high-precision 3D facial displacement field; the preset biomechanical model is used to perform constrained filtering on the facial displacement field to eliminate the deformation vectors that exceed the physiological limit and obtain the deformation trajectory. According to the pulsation time series signal, the enhanced texture feature map, and the deformation trajectory, in vivo determination is performed through time-frequency domain perturbation detection. Specifically, the time-frequency consistency between the pulsation time series signal and the deformation trajectory is calculated to obtain the physiological motion matching degree; the high-frequency singularity of the silicone artifact is detected using the enhanced texture feature map to obtain the material anomaly degree; the wavelet domain energy distribution difference between the near-infrared image sequence and the visible-light image sequence is calculated. If the phase consistency of the high-frequency subbands is lower than the preset threshold, it is determined that there is an adversarial noise attack and a live body alarm is issued; if the phase consistency of the high-frequency subbands is greater than or equal to the preset threshold, then when the physiological motion matching degree is less than the preset first threshold and the material anomaly degree is greater than the preset second threshold, a live body alarm is triggered. In this way, the present application provides a method that can effectively improve the accuracy and robustness of in vivo human face detection and overcomes various limitations existing in the prior art.

[0037] Optionally, the synchronous acquisition of binocular visible light and near-infrared images and cross-modal alignment to generate a sequence of binocular visible-light images and a sequence of near-infrared images that are pixel-level spatially corresponding includes:

[0038] Synchronously trigger the near-infrared camera and the visible-light binocular camera to obtain the facial near-infrared image and the binocular visible-light image at the same moment;

[0039] Specifically, strict synchronous acquisition of the binocular visible light camera and the near-infrared camera is achieved through a hardware synchronization signal (such as an external trigger circuit) or a software synchronization protocol (such as a global clock based on timestamps). The trigger signal needs to ensure that the exposure times of the two cameras completely overlap, so as to obtain the optical property difference image at the same moment and avoid time-domain misalignment. The near-infrared image (wavelength 850 - 940 nm) can penetrate the epidermal layer to capture subcutaneous microvascular information, while the visible light image (wavelength 400 - 700 nm) presents the skin surface texture. The synchronous alignment of the two provides spatio-temporal consistency guarantee for subsequent cross-modal feature fusion.

[0040] Cross-modal image alignment is performed through SURF feature point matching to generate a binocular visible light image sequence and a near-infrared image sequence with pixel-level spatial correspondence.

[0041] Specifically, SURF (Speeded-Up Robust Features) feature points are extracted from the near-infrared image and the visible light image respectively. Stable feature points are screened with the Hessian matrix threshold (such as setting the threshold to 500) to filter out the noise in the low-contrast area. A 64-dimensional descriptor is constructed based on the Haar wavelet response of direction normalization. The similarity of feature points is calculated through the Euclidean distance, and a two-way matching strategy (such as KNN matching + ratio test) is used to screen the candidate matching pairs. The RANSAC (Random Sample Consensus) algorithm is used to estimate the optimal affine transformation matrix, and the mis-matched points are removed (such as setting the reprojection error threshold to ±2 pixels) to align the pixel coordinate systems of the visible light image and the near-infrared image. The visible light image sequence is projected into the near-infrared image coordinate system through affine transformation and resampled using the bilinear interpolation method to ensure pixel-level spatial alignment.

[0042] Specifically, through the synchronous trigger and SURF feature point matching technologies, this application has successfully solved the problems of synchronous acquisition and cross-modal alignment of binocular visible light and near-infrared images. Compared with the prior art, the technical solution of this application has the following advantages: First, through the synchronous trigger mechanism, the time synchronization of image acquisition is ensured, avoiding the image mismatch problem caused by time differences. Second, through the SURF feature point matching technology, the spatial alignment of different modal images is achieved, generating an image sequence with pixel-level spatial correspondence, providing high-quality input data for subsequent image processing and analysis. Finally, the technical solution of this application simplifies the process of cross-modal image alignment, improving the accuracy and robustness of alignment.

[0043] Optionally, extracting the subcutaneous microvascular pulsation time series signal from the near-infrared image sequence includes:

[0044] Separating the subcutaneous scattering component from the near-infrared image sequence to obtain the first physiological feature map;

[0045] Eliminate the specular reflection noise of the first physiological feature map based on the Retinex algorithm to obtain a second physiological feature map;

[0046] Use the optical flow method to track the microvascular pulsation signal in the facial area of the second physiological feature map, extract the periodic blood flow change feature, and obtain a pulsation time series signal.

[0047] Specifically, by separating the subcutaneous scattering components of the near-infrared image sequence, the physiological features of subcutaneous microvessels can be effectively extracted. Using the Retinex algorithm to eliminate specular reflection noise can improve the quality of the physiological feature map and reduce noise interference. Finally, by using the optical flow method to track the microvascular pulsation signal in the facial area, extracting the periodic blood flow change feature, and obtaining a pulsation time series signal. These series of steps cooperate with each other to effectively solve the technical problem of extracting the subcutaneous microvascular pulsation time series signal from the near-infrared image sequence.

[0048] Exemplarily, the separation of subcutaneous scattering components can be achieved by various algorithms, such as the separation algorithm based on the scattering model, which can effectively separate the scattering components of subcutaneous microvessels. The Retinex algorithm is used to eliminate specular reflection noise, and its principle is to separate and balance the brightness and color of the image to eliminate the highlight noise caused by skin surface reflection. The optical flow method is used to track the microvascular pulsation signal, and the classic Lucas-Kanade optical flow method or the optical flow estimation method based on deep learning can be adopted. These methods can accurately capture the tiny blood flow change signal. By separating the subcutaneous scattering components of the near-infrared image sequence, eliminating specular reflection noise with the Retinex algorithm, and tracking the microvascular pulsation signal with the optical flow method, the present application can efficiently and accurately extract the pulsation time series signal of subcutaneous microvessels. Compared with the prior art, the method of the present application has significant advantages in terms of noise elimination, accuracy, and robustness of signal extraction, and can better meet the requirements of in vivo detection for subcutaneous physiological feature extraction.

[0049] Optionally, the combining the pulsation time series signal and the near-infrared image sequence to enhance the texture boundary feature in the visible light image sequence to obtain an enhanced texture feature map includes:

[0050] Calculate the microvascular pulsation frequency according to the phase difference of the pulsation time series signal, and generate a reflectivity sampling window aligned with the heartbeat cycle to obtain window parameters;

[0051] Calculate the reflectivity difference in the near-infrared band within the time interval defined by the window parameters to generate a material reflectivity anomaly map;

[0052] Perform edge detection on the visible light image to generate a gradient magnitude map;

[0053] Cascade the reflection anomaly map and the gradient magnitude map at the channel level, and input them into a spectral-sensitive convolutional layer to generate spatial attention weights;

[0054] Perform high-frequency filtering on the visible light image to obtain an edge texture feature map;

[0055] Multiply the attention weights and the edge texture feature map pixel by pixel to generate an enhanced texture feature map.

[0056] Exemplarily, first, it is necessary to calculate the microvascular pulsation frequency based on the phase difference of the pulsation timing signal, which can be achieved by analyzing the subcutaneous microvascular pulsation timing signal in the near-infrared image sequence. When generating the reflectivity sampling window aligned with the heartbeat cycle, the stability of the heartbeat cycle and the accuracy of the sampling window need to be considered. Within the time interval defined by the window parameters, calculate the reflectivity difference in the near-infrared band. The differential method or other reflectivity calculation methods can be used to generate the material reflection anomaly map. For edge detection of visible light images, edge detection algorithms such as Sobel operator and Canny operator can be used to generate the gradient magnitude map. When cascading the reflection anomaly map and the gradient magnitude map at the channel level, it is necessary to ensure the accurate spatial correspondence of the images and generate spatial attention weights through a spectral-sensitive convolutional layer. High-frequency filtering can use methods such as Fourier transform and Gaussian filtering to obtain the edge texture feature map. Finally, multiplying the attention weights and the edge texture feature map pixel by pixel can be achieved through pixel-by-pixel multiplication operations to generate an enhanced texture feature map. This application combines the near-infrared image sequence and the visible light image sequence, uses the phase difference calculation of the microvascular pulsation signal and the reflectivity sampling window aligned with the heartbeat cycle, and enhances the texture boundary features in the visible light image sequence. By calculating the reflectivity difference in the near-infrared band to generate the material reflection anomaly map and combining it with the gradient magnitude map of the visible light image, spatial attention weights are generated. Further, through high-frequency filtering and pixel-by-pixel multiplication operations, an enhanced texture feature map is generated. Thus, compared with the prior art, this application can more effectively enhance the texture boundary features in the visible light image, improving the accuracy and robustness of in-vivo detection.

[0057] Optionally, generating a facial depth map sequence based on the enhanced texture feature map, and analyzing the 3D deformation continuity and physiological motion consistency in the expression actions, the obtained deformation continuity analysis results include:

[0058] Based on the enhanced texture feature map, calculate an initial depth map based on binocular disparity;

[0059] Register the initial depth map with the facial key points of the near-infrared image sequence to obtain a high-precision 3D facial displacement field;

[0060] Constrained filtering is performed on the facial displacement field using a preset biomechanical model to eliminate deformation vectors that exceed the physiological limit, thereby obtaining a deformation trajectory.

[0061] Specifically, based on the enhanced texture feature map, an initial depth map is calculated based on binocular disparity; the initial depth map is registered with the facial key points of the near-infrared image sequence to obtain a high-precision 3D facial displacement field; constrained filtering is performed on the facial displacement field using a preset biomechanical model to eliminate deformation vectors that exceed the physiological limit, thereby obtaining a deformation trajectory. These features cooperate with each other to solve the problems of generating a facial depth map sequence based on a visible light image sequence and analyzing the 3D deformation continuity and physiological movement consistency in expression actions.

[0062] Exemplarily, based on the enhanced texture feature map, an initial depth map is calculated through binocular disparity. Specifically, two perspective images of the same scene are captured using a binocular camera, and the depth information of each pixel is calculated through disparity to generate an initial depth map. The initial depth map is registered with the facial key points of the near-infrared image sequence to obtain a high-precision 3D facial displacement field. The registration of facial key points can be achieved through a feature point matching algorithm, such as SURF feature point matching. Constrained filtering is performed on the facial displacement field using a preset biomechanical model to eliminate deformation vectors that exceed the physiological limit, thereby obtaining a deformation trajectory. The biomechanical model can be modeled based on the physical properties of facial muscles and skin, and the deformation vectors that do not conform to physiological characteristics are eliminated through filtering to ensure the authenticity of the deformation trajectory. Through the above technical features and steps, the present application can effectively solve the problems of generating a facial depth map sequence based on a visible light image sequence and analyzing the 3D deformation continuity and physiological movement consistency in expression actions in the prior art. Compared with the prior art, the present application provides a more accurate and reliable method for generating facial depth maps and analyzing deformations, which can improve the accuracy and robustness of live detection.

[0063] Optionally, the performing live detection through time-frequency domain perturbation detection according to the pulsation timing signal, the enhanced texture feature map, and the deformation continuity analysis result includes:

[0064] Calculating the time-frequency consistency between the pulsation timing signal and the deformation trajectory to obtain a physiological movement matching degree;

[0065] Detecting the high-frequency singularity of silicone artifacts using the enhanced texture feature map to obtain a material abnormality degree;

[0066] Calculating the wavelet domain energy distribution difference between the near-infrared image sequence and the visible light image sequence. If the phase consistency of the high-frequency subband is lower than a preset threshold, it is determined that there is an adversarial noise attack and a live alarm is issued;

[0067] If the phase consistency of the high-frequency sub-band is greater than or equal to a preset threshold, a live body alarm is triggered when the physiological motion matching degree is less than a preset first threshold and the material abnormality degree is greater than a preset second threshold.

[0068] Specifically, by calculating the time-frequency consistency between the pulsation time series signal and the deformation trajectory, the matching degree of physiological motion can be judged, so as to distinguish real physiological motion from forged mechanical motion. Using the enhanced texture feature map to detect the high-frequency singularity of silicone artifacts can identify the abnormal features of forged materials. By calculating the wavelet domain energy distribution difference between the near-infrared image sequence and the visible light image sequence, the presence of anti-noise attacks can be detected. The above technical features cooperate with each other to effectively solve the problems of anti-noise attacks and high-fidelity forgery attacks in live body detection, and improve the accuracy and robustness of live body detection.

[0069] Exemplarily, first, when calculating the time-frequency consistency between the pulsation time series signal and the deformation trajectory, a band-pass filtering method can be used to extract the main frequency features within the heartbeat frequency band, and perform low-dimensional motion mode decomposition on the deformation trajectory for power spectrum analysis. In the frequency band near the main heartbeat frequency, calculate the spectral energy ratio of each motion mode and the coherence with the microvascular signal, and fuse the energy ratio and coherence results to generate the physiological motion matching degree. When the physiological motion matching degree exceeds the preset threshold, it is determined that the physiological motion is consistent. Second, when using the enhanced texture feature map to detect the high-frequency singularity of silicone artifacts, multi-scale wavelet decomposition can be performed on the enhanced texture feature map to extract the local singularity points of the non-linear response in the high-frequency sub-band. Based on the time-frequency distribution characteristics of the singularity points, a frequency domain fingerprint template of the silicone material is constructed, and the time-frequency ridge line difference degree between the time-frequency distribution characteristics and the frequency domain fingerprint template is calculated. The kurtosis and skewness indexes of the difference degree in the spatial domain are statistically analyzed to generate the material abnormality degree, which characterizes the deviation degree of the reflection characteristics of the artifact material from the biological tissue. Finally, when calculating the wavelet domain energy distribution difference between the near-infrared image sequence and the visible light image sequence, a wavelet transform method can be used to decompose the image sequence and extract the phase information of the high-frequency sub-band. By calculating the phase consistency of the high-frequency sub-band, it is determined whether there is an anti-noise attack. When the phase consistency is lower than the preset threshold, a live body alarm is issued; when the phase consistency is greater than or equal to the preset threshold, further determination is made in combination with the physiological motion matching degree and the material abnormality degree. The present application effectively solves the technical problems of anti-noise attacks and high-fidelity forgery attacks in traditional methods by introducing a variety of technical means and performing live body detection from three aspects: physiological motion matching degree, material abnormality degree, and time-frequency domain energy distribution, and significantly improves the accuracy and robustness of live body detection.

[0070] Optionally, the calculating the time-frequency consistency between the pulsation time series signal and the deformation trajectory to obtain the physiological motion matching degree includes:

[0071] Perform time alignment and band-pass filtering on the pulsatile timing signal and the deformation trajectory, and respectively extract the main frequency characteristics within the heartbeat frequency band;

[0072] Decompose the 3D displacement field of the deformation trajectory into low-dimensional motion modes, and perform power spectrum analysis on each mode;

[0073] Within the frequency band near the heartbeat main frequency, calculate the spectral energy proportion of each motion mode and the coherence with the microvascular signal;

[0074] Fuse the results of the energy proportion and coherence to generate the physiological motion matching degree, and when the physiological motion matching degree exceeds a preset threshold, it is determined that the physiological motions are consistent.

[0075] Specifically, the technical features of the present application include performing time alignment and band-pass filtering on the pulsatile timing signal and the deformation trajectory, and respectively extracting the main frequency characteristics within the heartbeat frequency band; decomposing the 3D displacement field of the deformation trajectory into low-dimensional motion modes, and performing power spectrum analysis on each mode; within the frequency band near the heartbeat main frequency, calculating the spectral energy proportion of each motion mode and the coherence with the microvascular signal; fusing the results of the energy proportion and coherence to generate the physiological motion matching degree. Through these technical features, the problem of the consistency between the microvascular pulsation frequency and the deformation trajectory time frequency can be solved. Specifically, by performing time alignment and band-pass filtering to extract the main frequency characteristics within the heartbeat frequency band, decomposing the 3D displacement field of the deformation trajectory into low-dimensional motion modes, and performing power spectrum analysis, calculating the spectral energy proportion of each motion mode and the coherence with the microvascular signal, and finally fusing these results to generate the physiological motion matching degree, so as to judge the consistency of physiological motions.

[0076] Exemplarily, the steps of time-aligning and band-pass filtering the pulsation time series signal and the deformation trajectory can be implemented by a high-precision time synchronization device to ensure that the signals are aligned under the same time reference. The implementation of band-pass filtering can use digital filters, such as FIR filters or IIR filters, to extract the main frequency characteristics within the heartbeat frequency band. The 3D displacement field of the deformation trajectory can be decomposed into low-dimensional motion modes by dimensionality reduction techniques such as principal component analysis (PCA) or independent component analysis (ICA). Power spectrum analysis can use methods such as fast Fourier transform (FFT) or wavelet transform. In the frequency band near the main heartbeat frequency, calculating the spectral energy proportion of each motion mode and the coherence with the microvascular signal can be achieved through spectral analysis tools such as MATLAB or the SciPy library in Python. Fusing the results of the energy proportion and coherence to generate the physiological motion matching degree can be achieved through weighted averaging or other fusion algorithms. This application solves the problem of the consistency between the microvascular pulsation frequency and the deformation trajectory time frequency by time-aligning and band-pass filtering the pulsation time series signal and the deformation trajectory, extracting the main frequency characteristics within the heartbeat frequency band, decomposing the 3D displacement field of the deformation trajectory into low-dimensional motion modes, performing power spectrum analysis, calculating the spectral energy proportion of each motion mode and the coherence with the microvascular signal, and finally fusing these results to generate the physiological motion matching degree. Compared with the prior art, this application can more accurately judge the consistency of physiological motion and improve the accuracy and robustness of in-vivo detection.

[0077] Optionally, the detecting the high-frequency singularity of the silicone artifact using the enhanced texture feature map to obtain the material abnormality degree includes:

[0078] Performing multi-scale wavelet decomposition on the enhanced texture feature map to extract local singularity points of the non-linear response in the high-frequency sub-band;

[0079] Constructing a frequency-domain fingerprint template of the silicone material based on the time-frequency distribution characteristics of the singularity points, and calculating the time-frequency ridge line difference degree between the time-frequency distribution characteristics and the frequency-domain fingerprint template;

[0080] Statistically analyzing the kurtosis and skewness indexes of the difference degree in the spatial domain to generate the material abnormality degree, where the material abnormality degree characterizes the deviation degree of the reflection characteristics of the artifact material from the biological tissue.

[0081] Specifically, the high-frequency singularity of silicone artifacts is detected using the enhanced texture feature map; the enhanced texture feature map is subjected to multi-scale wavelet decomposition to extract local singularity points with non-linear responses in the high-frequency sub-bands; a frequency-domain fingerprint template of the silicone material is constructed based on the time-frequency distribution characteristics of the singularity points, and the time-frequency ridge line difference degree between the time-frequency distribution characteristics and the frequency-domain fingerprint template is calculated; the kurtosis and skewness indexes of the difference degree in the spatial domain are statistically analyzed to generate the material abnormality degree. These technical features cooperate with each other to solve the technical problem of detecting the high-frequency singularity of silicone artifacts through the enhanced texture feature map and obtaining the material abnormality degree. Specifically, by performing multi-scale wavelet decomposition on the enhanced texture feature map and extracting local singularity points with non-linear responses in the high-frequency sub-bands, the high-frequency singularity features of silicone artifacts can be effectively captured. Then, based on the time-frequency distribution characteristics of these singularity points, a frequency-domain fingerprint template of the silicone material is constructed, and the time-frequency ridge line difference degree between these characteristics and the template is calculated, further enhancing the detection ability for silicone artifacts. Finally, by statistically analyzing the kurtosis and skewness indexes of the difference degree in the spatial domain to generate the material abnormality degree, the deviation degree of the reflection characteristics between the artifact material and biological tissue can be characterized, improving the accuracy and robustness of in-vivo detection.

[0082] Exemplarily, the enhanced texture feature map is subjected to multi-scale wavelet decomposition to extract local singularity points with non-linear responses in the high-frequency sub-bands. The wavelet decomposition method can adopt discrete wavelet transform (DWT) or continuous wavelet transform (CWT), etc., to capture the details of different frequency components in the image through multi-scale analysis. A frequency-domain fingerprint template of the silicone material is constructed based on the time-frequency distribution characteristics of the singularity points, and the time-frequency ridge line difference degree between the time-frequency distribution characteristics and the frequency-domain fingerprint template is calculated. The frequency-domain fingerprint template can be obtained through training with pre-collected silicone artifact sample images, and the time-frequency ridge line difference degree can be measured by calculating the similarity between the time-frequency distribution characteristics and the frequency-domain fingerprint template. The kurtosis and skewness indexes of the difference degree in the spatial domain are statistically analyzed to generate the material abnormality degree. The kurtosis and skewness indexes can be calculated through statistical methods, such as fitting using a Gaussian distribution model, to obtain a quantitative index of the material abnormality degree. The technical solution of this application can effectively solve the problem of artifact detection in traditional in-vivo detection methods by using the enhanced texture feature map to detect the high-frequency singularity of silicone artifacts. Compared with the prior art, the advantages of this application are that through multi-scale wavelet decomposition and time-frequency analysis methods, the high-frequency singularity features of silicone artifacts can be captured more accurately, thereby improving the accuracy and robustness of detection. Further, by statistically analyzing the kurtosis and skewness indexes of the difference degree in the spatial domain to generate the material abnormality degree, the deviation degree of the reflection characteristics between the artifact material and biological tissue can be characterized more comprehensively, enhancing the reliability of in-vivo detection.

[0083] Optionally, the method further includes:

[0084] When the material abnormality degree is greater than the second threshold, reduce the reflectivity sampling window, and change the reflectivity sampling window from to , where , in the formula, is the sensitivity adjustment coefficient, is the material abnormality degree, is the second threshold;

[0085] Perform edge enhancement operations within the reduced window;

[0086] Calculate the reflectivity difference using the reduced window to obtain a new reflection anomaly map.

[0087] Specifically, when it is detected that the material abnormality degree is greater than a specific threshold, by reducing the reflectivity sampling window and performing edge enhancement operations within the reduced window, the reflectivity difference is further calculated to generate a new reflection anomaly map. Through the mutual cooperation of these technical features, when a relatively high material abnormality degree is detected, the reflection anomaly can be more accurately identified, thereby improving the accuracy of live detection. By reducing the reflectivity sampling window, the change in the reflection characteristics of the local area can be detected more concentratedly, reducing noise interference. Performing edge enhancement operations within the reduced window can capture edge features more clearly, further improving the accuracy of reflectivity difference calculation. Finally, a new reflection anomaly map is generated, which helps to more accurately determine the live situation.

[0088] Exemplarily, when the material abnormality degree is greater than the second threshold, the specific implementation method of reducing the reflectivity sampling window may include the following steps: First, according to the detected material abnormality degree C and the preset second threshold , calculate the adjusted coefficient , where is the sensitivity adjustment coefficient. Then, reduce the original reflectivity sampling window to . Within the reduced reflectivity sampling window, perform edge enhancement operations. Specifically, various edge detection algorithms can be used, such as Sobel, Canny, etc., to enhance edge features. Finally, use the reflectivity data within the reduced window to calculate the reflectivity difference and generate a new reflection anomaly map. The edge enhancement operation can adopt a gradient-based edge detection algorithm to further enhance edge features by calculating the image gradient magnitude map. Thus, the change in reflection characteristics can be captured more precisely, improving the accuracy of reflectivity difference calculation. Through the above technical solutions, when the material abnormality degree is relatively high, the present application can more accurately identify reflection anomalies by reducing the reflectivity sampling window and performing edge enhancement operations, thereby improving the accuracy of live detection. Compared with the prior art, the method of the present application can better cope with high-fidelity forgery attacks, reduce the missed detection rate, and improve the robustness and reliability of detection.

[0089] Based on the same inventive concept, such as Figure 3 shown, the present invention also provides a live face detection system based on a near-infrared and visible light binocular camera, and the system further includes:

[0090] A multi-modal module data processing module, configured to synchronously collect binocular visible light and near-infrared images and perform cross-modal alignment to generate a binocular visible light image sequence and a near-infrared image sequence with pixel-level spatial correspondence;

[0091] A physiological feature generation module, configured to extract the subcutaneous microvascular pulsation time series signal in the near-infrared image sequence;

[0092] A physical feature generation module, configured to combine the pulsation time series signal and the near-infrared image sequence to enhance the texture boundary features in the visible light image sequence and obtain an enhanced texture feature map;

[0093] A dynamic deformation verification module, configured to generate a facial depth map sequence based on the enhanced texture feature map and analyze the 3D deformation continuity and physiological movement consistency in the expression actions to obtain a deformation continuity analysis result;

[0094] A live body determination module, configured to perform live body determination through time-frequency domain perturbation detection according to the pulsation time series signal, the enhanced texture feature map, and the deformation continuity analysis result.

[0095] It should be noted that the electrical connections between the above-mentioned various units do not necessarily represent direct connections of the circuits. Indirect connection methods, as long as the purpose of the present invention is achieved, can be applied to the embodiments of the present invention. The above are only exemplary embodiments of the present invention and cannot be used to limit the scope of the present invention.

[0096] That is, any equivalent changes and modifications made in accordance with the teachings of the present invention still fall within the scope covered by the present invention. Those skilled in the art will easily think of other implementation schemes of the present invention after considering the specification and the disclosure of the practical truth. This application aims to cover any variations, uses, or adaptive changes of the present invention, and these variations, uses, or adaptive changes follow the general principles of the present invention and include the common general knowledge or conventional technical means in the technical field not recorded in the present invention.

Claims

1. A method for detecting a live human face based on a near-infrared and visible light binocular camera, characterized in that, The method includes: Synchronously collecting binocular visible light and near-infrared images and performing cross-modal alignment to generate a sequence of binocular visible light images and a sequence of near-infrared images with pixel-level spatial correspondence; Extracting the subcutaneous microvascular pulsation time series signal from the sequence of near-infrared images; Combining the pulsation time series signal and the sequence of near-infrared images to enhance the texture boundary features in the sequence of visible light images and obtaining an enhanced texture feature map; Calculating an initial depth map based on binocular disparity according to the enhanced texture feature map; Registering the initial depth map with the facial key points of the sequence of near-infrared images to obtain a high-precision 3D face displacement field; Using a preset biomechanical model to perform constrained filtering on the face displacement field, removing deformation vectors beyond the physiological limit, and obtaining a deformation trajectory; Performing a live detection through time-frequency domain perturbation detection according to the pulsation time series signal, the enhanced texture feature map, and the deformation trajectory; Among them, the combining the pulsation time series signal and the sequence of near-infrared images to enhance the texture boundary features in the sequence of visible light images and obtaining an enhanced texture feature map includes: Calculating the microvascular pulsation frequency according to the phase difference of the pulsation time series signal, generating a reflectivity sampling window aligned with the heartbeat cycle, and obtaining window parameters; Calculating the reflectivity difference in the near-infrared band within the time interval defined by the window parameters to generate a material reflectivity anomaly map; Performing edge detection on the visible light image to generate a gradient magnitude map; Performing channel concatenation on the reflectivity anomaly map and the gradient magnitude map and inputting them into a spectral-sensitive convolutional layer to generate spatial attention weights; Performing high-frequency filtering on the visible light image to obtain an edge texture feature map; Multiplying the attention weights and the edge texture feature map pixel by pixel to generate an enhanced texture feature map.

2. The live face detection method based on a near-infrared and visible light binocular camera according to claim 1, wherein The synchronously collecting binocular visible light and near-infrared images and performing cross-modal alignment to generate a sequence of binocular visible light images and a sequence of near-infrared images with pixel-level spatial correspondence includes: Synchronously triggering a near-infrared camera and a binocular visible light camera to obtain a facial near-infrared image and a binocular visible light image at the same moment; Performing cross-modal image alignment through SURF feature point matching to generate a sequence of binocular visible light images and a sequence of near-infrared images with pixel-level spatial correspondence.

3. The method for detecting a living human face based on a near-infrared and visible light binocular camera according to claim 1, wherein, The extracting the subcutaneous microvascular pulsation time series signal from the sequence of near-infrared images includes: Separating the subcutaneous scattering component from the sequence of near-infrared images to obtain a first physiological feature map; Eliminating the specular reflection noise of the first physiological feature map based on the Retinex algorithm to obtain a second physiological feature map; Adopting an optical flow method to track the microvascular pulsation signal in the facial area of the second physiological feature map, extracting the periodic blood flow change feature, and obtaining the pulsation time series signal.

4. A method for detecting a live human face based on a near-infrared and visible light binocular camera according to claim 1, characterized in that, The performing a live detection through time-frequency domain perturbation detection according to the pulsation time series signal, the enhanced texture feature map, and the deformation trajectory includes: Calculating the time-frequency consistency between the pulsation time series signal and the deformation trajectory to obtain a physiological motion matching degree; Detecting the high-frequency singularity of the silicone artifact using the enhanced texture feature map to obtain a material anomaly degree; Calculate the difference in wavelet domain energy distribution between the near-infrared image sequence and the visible light image sequence. If the phase consistency of the high-frequency subband is lower than a preset threshold, it is determined that there is an adversarial noise attack and a live body alarm is issued; If the phase consistency of the high-frequency subband is greater than or equal to the preset threshold, then when the physiological motion matching degree is less than a preset first threshold and the material abnormality degree is greater than a preset second threshold, a live body alarm is triggered.

5. A method for detecting a living human face based on a near-infrared and visible-light binocular camera according to claim 4, characterized in that The calculating the time-frequency consistency between the pulsation time series signal and the deformation trajectory to obtain the physiological motion matching degree includes: Perform time alignment and band-pass filtering on the pulsation time series signal and the deformation trajectory, and respectively extract the main frequency features within the heartbeat frequency band; Decompose the 3D displacement field of the deformation trajectory into low-dimensional motion modes, and perform power spectrum analysis on each mode; Within the frequency band near the heartbeat main frequency, calculate the spectral energy proportion of each motion mode and the coherence with the microvascular signal; Fuse the energy proportion and coherence results to generate the physiological motion matching degree, where when the physiological motion matching degree exceeds the preset threshold, it is determined that the physiological motion is consistent.

6. The live face detection method based on a near-infrared and visible light binocular camera according to claim 4, characterized in that The detecting the high-frequency singularity of the silicone artifact using the enhanced texture feature map to obtain the material abnormality degree includes: Perform multi-scale wavelet decomposition on the enhanced texture feature map, and extract the local singularity points of the non-linear response in the high-frequency subband; Based on the time-frequency distribution characteristics of the singularity points, construct a frequency-domain fingerprint template of the silicone material, and calculate the time-frequency ridge line difference degree between the time-frequency distribution characteristics and the frequency-domain fingerprint template; Statistically analyze the kurtosis and skewness indexes of the difference degree in the spatial domain to generate the material abnormality degree, where the material abnormality degree characterizes the deviation degree of the reflection characteristics of the artifact material from the biological tissue.

7. A live face detection method based on a near-infrared and visible light binocular camera according to claim 4, characterized in that, The method further includes: When the material abnormality degree is greater than the second threshold, reduce the reflectivity sampling window, and change the reflectivity sampling window from to , where . In the formula, is the sensitivity adjustment coefficient, is the material abnormality degree, is the second threshold; Perform edge enhancement operation within the reduced window; Use the reduced window to calculate the reflectivity difference to obtain a new reflection anomaly map.

8. A live face detection system based on a near-infrared and visible light binocular camera, characterized in that, The system includes: A multi-modal module data processing module, configured to synchronously collect binocular visible light and near-infrared images and perform cross-modal alignment to generate a binocular visible light image sequence and a near-infrared image sequence with pixel-level spatial correspondence; A physiological feature generation module, configured to extract the subcutaneous microvascular pulsation time series signal from the near-infrared image sequence; A physical feature generation module, which is used to combine the pulsation timing signal and the near-infrared image sequence to enhance the texture boundary features in the visible light image sequence and obtain an enhanced texture feature map; wherein, the combining the pulsation timing signal and the near-infrared image sequence to enhance the texture boundary features in the visible light image sequence and obtain an enhanced texture feature map includes: calculating the microvascular pulsation frequency according to the phase difference of the pulsation timing signal, generating a reflectivity sampling window aligned with the heartbeat cycle, and obtaining window parameters; calculating the reflectivity difference in the near-infrared band within the time interval defined by the window parameters to generate a material reflection anomaly map; performing edge detection on the visible light image to generate a gradient magnitude map; performing channel concatenation on the reflection anomaly map and the gradient magnitude map and inputting them into a spectral-sensitive convolutional layer to generate spatial attention weights; performing high-frequency filtering on the visible light image to obtain an edge texture feature map; multiplying the attention weights and the edge texture feature map pixel by pixel to generate an enhanced texture feature map; A dynamic deformation verification module, which is used to calculate an initial depth map based on binocular disparity according to the enhanced texture feature map; register the initial depth map with the facial key points of the near-infrared image sequence to obtain a high-precision 3D face displacement field; and use a preset biomechanical model to perform constrained filtering on the face displacement field to remove deformation vectors beyond the physiological limit and obtain a deformation trajectory; A living body judgment module, which is used to perform living body judgment through time-frequency domain perturbation detection according to the pulsation timing signal, the enhanced texture feature map, and the deformation trajectory.

Images