Living body face detection method based on near infrared and visible light binocular camera

Abstract

The invention discloses a living body face detection method based on a near-infrared and visible light binocular camera, and belongs to the technical field of face recognition, and the method comprises the steps: collecting binocular visible light and near-infrared images, and carrying out the cross-modal alignment; subcutaneous microvascular pulsation time sequence signals in the near-infrared image are extracted, texture boundary features in the visible light image sequence are enhanced in combination with the near-infrared image sequence, and an enhanced texture feature map is obtained; analyzing 3D deformation continuity and physiological motion consistency in the expressions and actions based on the enhanced texture feature map; and in combination with time-frequency domain disturbance detection, carrying out living body determination. According to the method, a multi-modal time sequence synchronous material reflection gradient analysis and microvascular pulsation time sequence tracking technology is adopted, and in combination with facial dynamic 3D deformation biomechanical model constraint and time-frequency combined anti-interference verification, the accuracy and robustness of in-vivo detection can be effectively improved, and a powerful guarantee is provided for the safety of a face recognition technology.

Description

Technical Field

[0001] The present invention relates to the technical field of facial recognition, and in particular to a living human face detection method based on near-infrared and visible light binocular cameras. Background Art

[0002] With the widespread application of face recognition technology, liveness detection technology has become a key link in ensuring the security of face recognition. Traditional liveness detection methods mainly rely on a single image modality, such as visible light images. These methods are easily affected by lighting conditions, image quality, and forgery methods, such as photos, videos, or 3D printed masks, resulting in insufficient accuracy and robustness of liveness detection.

[0003] Currently existing liveness detection methods rely on single-modality solutions, which have a high missed detection rate for high-simulation counterfeit attacks due to physiological blind spots and material reflection noise interference; the spatiotemporal coordination ability of multimodal data is weak, and cross-modal image alignment relies on manual calibration rather than dynamic feature alignment, which weakens the joint identification effect of abnormal silicone reflection and natural skin texture; dynamic deformation detection lacks biomechanical constraints, and monocular eye flow tracking cannot distinguish between real skin elastic deformation and the mechanical movement mode of the counterfeit body, making it difficult to resist dynamic counterfeit attacks; the time-frequency domain analysis method is not sensitive enough to the high-frequency singularity of the material, and has poor robustness against noise attacks, and frequency domain fingerprint matching is prone to failure. Summary of the invention

[0004] To solve the above problems, the present invention provides a live face detection method based on near-infrared and visible light binocular cameras, which adopts multi-modal time-series synchronized material reflection gradient analysis and microvascular pulsation timing tracking technology, combined with biomechanical model constraints of dynamic 3D deformation of the face and time-frequency joint anti-interference verification, which can effectively improve the accuracy and robustness of liveness detection and provide a strong guarantee for the security of face recognition technology.

[0005] The above objectives can be achieved through the following solutions: A method for detecting living faces based on near-infrared and visible light binocular cameras comprises: synchronously collecting binocular visible light and near-infrared images and performing cross-modal alignment to generate binocular visible light image sequences and near-infrared image sequences corresponding to pixel-level space; extracting subcutaneous microvascular pulsation timing signals in the near-infrared image sequence; combining the pulsation timing signals with the near-infrared image sequence, enhancing texture boundary features in the visible light image sequence, and obtaining an enhanced texture feature map; generating a facial depth map sequence based on the enhanced texture feature map, and analyzing 3D deformation continuity and physiological movement consistency in facial expressions and actions to obtain deformation continuity analysis results; and performing liveness determination through time-frequency domain perturbation detection according to the pulsation timing signals, the enhanced texture feature map, and the deformation continuity analysis results.

[0006] Optionally, the synchronous acquisition of binocular visible light and near-infrared images and cross-modal alignment to generate a binocular visible light image sequence and a near-infrared image sequence corresponding to the pixel-level space includes: synchronously triggering a near-infrared camera and a visible light binocular camera to obtain a facial near-infrared image and a binocular visible light image at the same moment; performing cross-modal image alignment through SURF feature point matching to generate a binocular visible light image sequence and a near-infrared image sequence corresponding to the pixel-level space.

[0007] Optionally, the extraction of subcutaneous microvascular pulsation timing signals in the near-infrared image sequence includes: separating the subcutaneous scattering components of the near-infrared image sequence to obtain a first physiological characteristic map; eliminating the mirror reflection noise of the first physiological characteristic map based on the Retinex algorithm to obtain a second physiological characteristic map; using the optical flow method to track the microvascular pulsation signal in the facial area of ​​the second physiological characteristic map, extracting periodic blood flow change characteristics, and obtaining a pulsation timing signal.

[0008] Optionally, the method of enhancing the texture boundary features in the visible light image sequence in combination with the near-infrared image sequence to obtain an enhanced texture feature map includes: calculating the microvascular pulsation frequency based on the phase difference of the pulsation timing signal, and generating a reflectivity sampling window aligned with the heartbeat cycle to obtain window parameters; calculating the reflectivity difference in the near-infrared band within the time interval defined by the window parameters to generate a material reflection anomaly map; performing edge detection on the visible light image to generate a gradient amplitude map; channel-concatenating the reflection anomaly map and the gradient amplitude map, and inputting them into a spectrally sensitive convolutional layer to generate spatial attention weights; performing high-frequency filtering on the visible light image to obtain an edge texture feature map; and multiplying the attention weights by the edge texture feature map pixel by pixel to generate an enhanced texture feature map.

[0009] Optionally, the facial depth map sequence is generated based on the enhanced texture feature map, and the 3D deformation continuity and physiological movement consistency in the facial expression are analyzed to obtain the deformation continuity analysis result, including: obtaining an initial depth map based on binocular parallax calculation according to the enhanced texture feature map; aligning the initial depth map with the facial key points of the near-infrared image sequence to obtain a high-precision 3D facial displacement field; and using a preset biomechanical model to constrain filtering the facial displacement field to eliminate deformation vectors that exceed physiological limits to obtain a deformation trajectory.

[0010] Optionally, the liveness determination through time-frequency domain perturbation detection based on the pulsation timing signal, the enhanced texture feature map, and the deformation continuity analysis result includes: calculating the time-frequency consistency between the pulsation timing signal and the deformation trajectory to obtain the physiological movement matching degree; using the enhanced texture feature map to detect the high-frequency singularity of the silicone artifact to obtain the material abnormality; calculating the difference in wavelet domain energy distribution between the near-infrared image sequence and the visible light image sequence, if the phase consistency of the high-frequency sub-band is lower than a preset threshold, determining that there is an anti-noise attack and issuing a liveness alarm; if the phase consistency of the high-frequency sub-band is greater than or equal to the preset threshold, then when the physiological movement matching degree is less than a preset first threshold and the material abnormality is greater than a preset second threshold, a liveness alarm is triggered.

[0011] Optionally, the calculation of the time-frequency consistency between the microvascular pulsation frequency and the deformation trajectory to obtain the physiological motion matching degree includes: time alignment and bandpass filtering of the pulsation timing signal and the deformation trajectory, and extracting the main frequency characteristics within the heartbeat frequency band respectively; decomposing the 3D displacement field of the deformation trajectory into low-dimensional motion modes, and performing power spectrum analysis on each mode; within the frequency band near the main frequency of the heartbeat, calculating the spectral energy proportion of each motion mode and the coherence with the microvascular signal; fusing the energy proportion and coherence results to generate the physiological motion matching degree, wherein the physiological motion matching degree is determined to be consistent when it exceeds a preset threshold.

[0012] Optionally, the use of the enhanced texture feature map to detect the high-frequency singularity of silicone artifacts to obtain the material abnormality includes: performing multi-scale wavelet decomposition on the enhanced texture feature map to extract local singularity points of nonlinear response in high-frequency sub-bands; constructing a frequency domain fingerprint template of the silicone material based on the time-frequency distribution characteristics of the singularity points, and calculating the difference in time-frequency ridges between the time-frequency distribution characteristics and the frequency domain fingerprint template; and calculating the kurtosis and skewness indicators of the difference in the spatial domain to generate the material abnormality, wherein the material abnormality characterizes the degree of deviation of the reflectance characteristics of the artifact material from the biological tissue.

[0013] Optionally, the method further comprises: when the material abnormality is greater than the second threshold, reducing the reflectivity sampling window from Reduce to ,in , where is the sensitivity adjustment coefficient, is the abnormality of the material, is the second threshold; performing edge enhancement operation in the reduced window; and calculating the reflectivity difference using the reduced window to obtain a new reflection anomaly map.

[0014] Based on the same inventive concept, the present invention also provides a live face detection system based on near-infrared and visible light binocular cameras, the system also including: a multimodal module data processing module, used to synchronously collect binocular visible light and near-infrared images and perform cross-modal alignment to generate a binocular visible light image sequence and a near-infrared image sequence corresponding to the pixel-level space; a physiological feature generation module, used to extract the subcutaneous microvascular pulsation timing signal in the near-infrared image sequence; a physical feature generation module, used to combine the pulsation timing signal and the near-infrared image sequence, enhance the texture boundary features in the visible light image sequence, and obtain an enhanced texture feature map; a dynamic deformation verification module, which generates a facial depth map sequence based on the enhanced texture feature map, and analyzes the 3D deformation continuity and physiological movement consistency in the expression action to obtain a deformation continuity analysis result; a live judgment module, which is used to perform live judgment through time-frequency domain perturbation detection based on the pulsation timing signal, the enhanced texture feature map, and the deformation continuity analysis result.

[0015] Compared with the prior art, the present invention has the following advantages: 1. The present invention synchronously collects binocular visible light and near-infrared images and combines the information of these two modalities for liveness detection. Near-infrared images are sensitive to physiological characteristics such as subcutaneous microvessels and can capture the unique pulsation signals of living bodies; while visible light images can present the texture details and facial expressions of the human face. This multi-modal fusion method effectively improves the accuracy of liveness detection and reduces the misjudgment that may be caused by a single modality; 2. The present invention uses SURF feature point matching to perform cross-modal image alignment and generate binocular visible light image sequences and near-infrared image sequences that correspond to pixel-level space. This technology ensures accurate alignment between images of different modalities, improves the robustness of subsequent feature extraction and analysis, and enables the system to work stably under different lighting conditions and complex backgrounds; 3. The present invention not only extracts the subcutaneous microvascular pulsation time series signal in the near-infrared image as a physiological feature, but also enhances the texture boundary feature in the visible light image by combining the signal to obtain an enhanced texture feature map. This method of combining physiological features with physical features further enhances the system's identification ability, enabling it to more accurately identify the difference between real faces and forged faces; 4. When analyzing the 3D deformation continuity and physiological movement consistency in facial expressions, the present invention uses a preset biomechanical model to constrain the facial displacement field and remove deformation vectors that exceed the physiological limit. This method effectively improves the accuracy of deformation analysis and reduces misjudgment caused by mechanical movement of forgeries; 5. The present invention uses multiple methods to determine liveness, such as calculating the temporal frequency consistency between the pulsation timing signal and the deformation trajectory, using enhanced texture feature maps to detect the high-frequency singularity of silicone artifacts, and calculating the difference in wavelet domain energy distribution between near-infrared and visible light image sequences. These time-frequency domain disturbance detection methods can effectively resist noise attacks and other interference factors, improving the system's anti-interference ability and overall performance.

[0016] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a flow chart of a method for detecting living faces based on near-infrared and visible light binocular cameras according to an embodiment of the present invention.

[0019] Figure 2 It is an execution flow chart of a living face detection method based on near-infrared and visible light binocular cameras according to an embodiment of the present invention.

[0020] Figure 3 It is a structural schematic diagram of a living face detection system based on near-infrared and visible light binocular cameras according to an embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] Reference Figure 1An embodiment of the present invention proposes a method for live face detection based on near-infrared and visible light binocular cameras, which adopts multi-modal time-series synchronized material reflection gradient analysis and microvascular pulsation time-series tracking technology, combined with biomechanical model constraints of dynamic 3D facial deformation and time-frequency joint anti-interference verification, which can effectively improve the accuracy and robustness of liveness detection and provide a strong guarantee for the security of face recognition technology.

[0023] The method of this embodiment specifically includes: Synchronously collect binocular visible light and near-infrared images and perform cross-modal alignment to generate binocular visible light image sequences and near-infrared image sequences that correspond to each other at the pixel level. Extracting subcutaneous microvascular pulsation timing signals from the near-infrared image sequence; Combining the pulsation timing signal and the near-infrared image sequence, enhancing the texture boundary features in the visible light image sequence to obtain an enhanced texture feature map; Generate a facial depth map sequence based on the enhanced texture feature map, and analyze the 3D deformation continuity and physiological movement consistency in the facial expression to obtain a deformation continuity analysis result; Based on the pulsation timing signal, the enhanced texture feature map, and the deformation continuity analysis result, liveness determination is performed through time-frequency domain disturbance detection.

[0024] Specifically, by synchronously collecting binocular visible light and near-infrared images and performing cross-modal alignment, the problem that single-modal images are easily affected by lighting conditions, image quality, and counterfeiting methods is solved. By extracting the subcutaneous microvascular pulsation timing signal in the near-infrared image sequence and combining the pulsation timing signal with the near-infrared image sequence, the texture boundary features in the visible light image sequence are enhanced, solving the problems of physiological feature blind spots and material reflection noise interference. Based on the enhanced texture feature map, a facial depth map sequence is generated, and the 3D deformation continuity and physiological movement consistency in facial expressions are analyzed, solving the problem of lack of biomechanical constraints in dynamic deformation detection. Liveness determination is performed through time-frequency domain perturbation detection, solving the problems of insufficient sensitivity to high-frequency singularities of materials and poor robustness against noise attacks.

[0025] For example, Figure 2As shown in the figure, firstly, by synchronously triggering the near-infrared camera and the visible light binocular camera, the near-infrared image and binocular visible light image of the face at the same time are obtained. Then, the cross-modal image alignment is performed by matching the SURF feature points to generate the binocular visible light image sequence and the near-infrared image sequence corresponding to the pixel-level space. On this basis, the subcutaneous scattering component of the near-infrared image sequence is separated to obtain the first physiological feature map; the mirror reflection noise of the first physiological feature map is eliminated based on the Retinex algorithm to obtain the second physiological feature map; the optical flow method is used to track the microvascular pulsation signal of the facial area in the second physiological feature map, and the periodic blood flow change characteristics are extracted to obtain the pulsation timing signal. The microvascular pulsation frequency is calculated according to the phase difference of the pulsation timing signal, and the reflectivity sampling window aligned with the heartbeat cycle is generated to obtain the window parameters. In the time interval defined by the window parameters, the reflectivity difference of the near-infrared band is calculated to generate the material reflection anomaly map. The visible light image is edge detected to generate the gradient amplitude map; the reflection anomaly map and the gradient amplitude map are channel-concatenated and input into the spectrally sensitive convolution layer to generate the spatial attention weight. Perform high-frequency filtering on the visible light image to obtain the edge texture feature map; multiply the attention weight and the edge texture feature map pixel by pixel to generate the enhanced texture feature map. Based on the enhanced texture feature map, the initial depth map is calculated using binocular disparity; the initial depth map is aligned with the facial key points of the near-infrared image sequence to obtain a high-precision 3D facial displacement field; the facial displacement field is constrained and filtered using a preset biomechanical model to remove deformation vectors that exceed the physiological limit and obtain the deformation trajectory. Based on the pulsation timing signal, enhanced texture feature map, and deformation trajectory, liveness determination is performed through time-frequency domain perturbation detection. Specifically, the temporal frequency consistency between the pulsation timing signal and the deformation trajectory is calculated to obtain the physiological motion matching degree; the high-frequency singularity of the silicone artifact is detected using the enhanced texture feature map to obtain the material abnormality; the difference in wavelet domain energy distribution between the near-infrared image sequence and the visible light image sequence is calculated. If the phase consistency of the high-frequency sub-band is lower than the preset threshold, it is determined that there is an anti-noise attack and a live alarm is issued; if the phase consistency of the high-frequency sub-band is greater than or equal to the preset threshold, then when the physiological motion matching degree is less than the preset first threshold and the material abnormality is greater than the preset second threshold, a live alarm is triggered. In this way, the present application provides a method that can effectively improve the accuracy and robustness of live face detection, overcoming the various limitations existing in the prior art.

[0026] Optionally, the synchronously collecting binocular visible light and near-infrared images and performing cross-modal alignment to generate a binocular visible light image sequence and a near-infrared image sequence corresponding to the pixel-level space includes: Synchronously trigger the near-infrared camera and the visible light binocular camera to obtain the facial near-infrared image and binocular visible light image at the same time; Specifically, strict synchronization of binocular visible light camera and near-infrared camera is achieved through hardware synchronization signals (such as external trigger circuits) or software synchronization protocols (such as global clocks based on timestamps). The trigger signal needs to ensure that the exposure time of the two cameras completely overlaps, so as to obtain images with different optical characteristics at the same time and avoid time domain misalignment. Near-infrared images (wavelength 850-940 nm) can penetrate the epidermis to capture subcutaneous microvascular information, while visible light images (wavelength 400-700nm) present the skin surface texture. The synchronous alignment of the two provides temporal and spatial consistency for subsequent cross-modal feature fusion.

[0027] Cross-modal image alignment is performed through SURF feature point matching to generate binocular visible light image sequences and near-infrared image sequences that correspond to each other at the pixel level.

[0028] Specifically, SURF (Speeded-Up Robust Features) feature points are extracted from the near-infrared image and the visible light image respectively, and the stable feature points are screened by the Hessian matrix threshold (such as the threshold is set to 500) to filter the noise in the low-contrast area. A 64-dimensional descriptor is constructed based on the direction-normalized Haar wavelet response, and the similarity of the feature points is calculated by the Euclidean distance. The candidate matching pairs are screened by a two-way matching strategy (such as KNN matching + ratio test). The RANSAC (random sampling consensus) algorithm is used to estimate the optimal affine transformation matrix, and the mismatched points are eliminated (such as setting the reprojection error threshold to ±2 pixels) to align the pixel coordinate systems of the visible light image and the near-infrared image. The visible light image sequence is projected to the near-infrared image coordinate system through affine transformation, and resampled using bilinear interpolation to ensure pixel-level spatial alignment.

[0029] Specifically, the present application successfully solves the problem of synchronous acquisition and cross-modal alignment of binocular visible light and near-infrared images through synchronous triggering and SURF feature point matching technology. Compared with the prior art, the technical solution of the present application has the following advantages: First, through the synchronous triggering mechanism, the time synchronization of image acquisition is ensured, and the image mismatch problem caused by time difference is avoided. Secondly, through the SURF feature point matching technology, the spatial alignment of images of different modalities is realized, and an image sequence corresponding to the pixel-level space is generated, which provides high-quality input data for subsequent image processing and analysis. Finally, the technical solution of the present application simplifies the process of cross-modal image alignment and improves the accuracy and robustness of the alignment.

[0030] Optionally, extracting the subcutaneous microvascular pulsation timing signal in the near-infrared image sequence includes: Separating subcutaneous scattering components of the near-infrared image sequence to obtain a first physiological characteristic map; Eliminating the specular reflection noise of the first physiological characteristic map based on the Retinex algorithm to obtain a second physiological characteristic map; The optical flow method is used to track the microvascular pulsation signal in the facial area in the second physiological characteristic image, extract the periodic blood flow change characteristics, and obtain the pulsation timing signal.

[0031] Specifically, by separating the subcutaneous scattering components of the near-infrared image sequence, the physiological characteristics of the subcutaneous microvessels can be effectively extracted. The use of the Retinex algorithm to eliminate mirror reflection noise can improve the quality of the physiological feature map and reduce noise interference. Finally, the microvascular pulsation signal in the facial area is tracked by the optical flow method, the periodic blood flow change characteristics are extracted, and the pulsation timing signal is obtained. This series of steps cooperates with each other and can effectively solve the technical problem of extracting the subcutaneous microvascular pulsation timing signal from the near-infrared image sequence.

[0032] Exemplarily, the separation of subcutaneous scattering components can be achieved through a variety of algorithms, such as a separation algorithm based on a scattering model, which can effectively separate the scattering components of subcutaneous microvessels. The Retinex algorithm is used to eliminate specular reflection noise. Its principle is to separate and balance the brightness and color of the image to eliminate the high light noise caused by the reflection of the skin surface. The optical flow method is used to track the microvascular pulsation signal. The classic Lucas-Kanade optical flow method or the optical flow estimation method based on deep learning can be used. These methods can accurately capture tiny blood flow change signals. The present application can efficiently and accurately extract the pulsation timing signal of subcutaneous microvessels by separating the subcutaneous scattering components of the near-infrared image sequence, eliminating specular reflection noise with the Retinex algorithm, and tracking the microvascular pulsation signal with the optical flow method. Compared with the prior art, the method of the present application has significant advantages in noise elimination, accuracy and robustness of signal extraction, and can better meet the needs of subcutaneous physiological feature extraction for liveness detection.

[0033] Optionally, the step of enhancing the texture boundary features in the visible light image sequence in combination with the near infrared image sequence to obtain an enhanced texture feature map comprises: Calculating the microvascular pulsation frequency according to the phase difference of the pulsation timing signal, and generating a reflectivity sampling window aligned with the heartbeat cycle to obtain a window parameter; In the time interval defined by the window parameters, the reflectivity difference in the near infrared band is calculated to generate a material reflection anomaly map; Performing edge detection on the visible light image to generate a gradient amplitude map; Perform channel concatenation on the reflection anomaly map and the gradient amplitude map, and input them into a spectrally sensitive convolutional layer to generate a spatial attention weight; Performing high-frequency filtering on the visible light image to obtain an edge texture feature map; The attention weight is multiplied pixel by pixel by the edge texture feature map to generate an enhanced texture feature map.

[0034] Exemplarily, it is first necessary to calculate the microvascular pulsation frequency according to the phase difference of the pulsation timing signal, which can be achieved by analyzing the subcutaneous microvascular pulsation timing signal in the near-infrared image sequence. When generating a reflectivity sampling window aligned with the heartbeat cycle, it is necessary to take into account the stability of the heartbeat cycle and the accuracy of the sampling window. In the time interval defined by the window parameters, the reflectivity difference in the near-infrared band is calculated, and the difference method or other reflectivity calculation methods can be used to generate a material reflection anomaly map. For edge detection of visible light images, edge detection algorithms such as Sobel operator and Canny operator can be used to generate a gradient amplitude map. When channel cascading the reflection anomaly map and the gradient amplitude map, it is necessary to ensure that the spatial correspondence of the image is accurate, and generate spatial attention weights through a spectrally sensitive convolution layer. High-frequency filtering can use Fourier transform, Gaussian filtering and other methods to obtain edge texture feature maps. Finally, the attention weight is multiplied pixel by pixel with the edge texture feature map, which can be achieved by pixel-by-pixel multiplication operation to generate an enhanced texture feature map. This application combines a near-infrared image sequence with a visible light image sequence, and utilizes the phase difference calculation of the microvascular pulsation signal and the reflectivity sampling window aligned with the heartbeat cycle to enhance the texture boundary features in the visible light image sequence. A material reflection anomaly map is generated by calculating the reflectivity difference in the near-infrared band, and the spatial attention weight is generated in combination with the gradient amplitude map of the visible light image. Furthermore, an enhanced texture feature map is generated through high-frequency filtering and pixel-by-pixel multiplication operations. Therefore, compared with the prior art, this application can more effectively enhance the texture boundary features in visible light images and improve the accuracy and robustness of liveness detection.

[0035] Optionally, the generating of a facial depth map sequence based on the enhanced texture feature map, and analyzing the 3D deformation continuity and physiological movement consistency in the facial expression action to obtain the deformation continuity analysis result includes: According to the enhanced texture feature map, an initial depth map is obtained based on binocular disparity calculation; Registering the initial depth map with the facial key points of the near infrared image sequence to obtain a high-precision 3D facial displacement field; The facial displacement field is constrained and filtered using a preset biomechanical model to remove deformation vectors that exceed physiological limits and obtain a deformation trajectory.

[0036] Specifically, based on the enhanced texture feature map, the initial depth map is obtained based on binocular disparity calculation; the initial depth map is aligned with the facial key points of the near-infrared image sequence to obtain a high-precision 3D facial displacement field; the facial displacement field is constrained and filtered using a preset biomechanical model to remove deformation vectors that exceed the physiological limit and obtain the deformation trajectory. These features work together to solve the problem of generating facial depth map sequences based on visible light image sequences and analyzing the 3D deformation continuity and physiological movement consistency in facial expressions.

[0037] Exemplarily, based on the enhanced texture feature map, an initial depth map is obtained by binocular disparity calculation. Specifically, two perspective images of the same scene are taken by a binocular camera, and the depth information of each pixel is obtained by disparity calculation, thereby generating an initial depth map. The initial depth map is aligned with the facial key points of the near-infrared image sequence to obtain a high-precision 3D facial displacement field. The alignment of facial key points can be achieved by a feature point matching algorithm, such as SURF feature point matching. The facial displacement field is constrained and filtered using a preset biomechanical model to remove deformation vectors that exceed the physiological limit and obtain a deformation trajectory. The biomechanical model can be modeled based on the physical properties of facial muscles and skin, and deformation vectors that do not conform to physiological characteristics are removed by filtering to ensure the authenticity of the deformation trajectory. The present application, through the above-mentioned technical features and steps, can effectively solve the problem of generating a facial depth map sequence based on a visible light image sequence in the prior art, and analyzing the 3D deformation continuity and physiological movement consistency in facial expressions. Compared with the prior art, the present application provides a more accurate and reliable facial depth map generation and deformation analysis method, which can improve the accuracy and robustness of liveness detection.

[0038] Optionally, the performing living body determination by time-frequency domain disturbance detection according to the pulsation timing signal, the enhanced texture feature map, and the deformation continuity analysis result includes: Calculating the temporal frequency consistency between the pulsation timing signal and the deformation trajectory to obtain a physiological motion matching degree; The enhanced texture feature map is used to detect the high-frequency singularity of the silicone artifact to obtain the material abnormality; Calculating the difference in wavelet domain energy distribution between the near-infrared image sequence and the visible light image sequence, if the phase consistency of the high-frequency sub-band is lower than a preset threshold, determining that there is an anti-noise attack and issuing a live alarm; If the phase consistency of the high-frequency sub-band is greater than or equal to a preset threshold, a living body alarm is triggered when the physiological movement matching degree is less than a preset first threshold and the material abnormality degree is greater than a preset second threshold.

[0039] Specifically, by calculating the temporal frequency consistency between the pulsation timing signal and the deformation trajectory, the matching degree of physiological movement can be determined, thereby distinguishing between real physiological movement and forged mechanical movement. By using enhanced texture feature maps to detect the high-frequency singularity of silicone artifacts, abnormal features of forged materials can be identified. By calculating the difference in wavelet domain energy distribution between near-infrared image sequences and visible light image sequences, the presence of anti-noise attacks can be detected. The above technical features work together to effectively solve the problems of anti-noise attacks and high-simulation forgery attacks in liveness detection, and improve the accuracy and robustness of liveness detection.

[0040] Exemplarily, first, when calculating the time-frequency consistency between the pulsation timing signal and the deformation trajectory, the bandpass filtering method can be used to extract the main frequency characteristics within the heartbeat frequency band, and the deformation trajectory is decomposed into a low-dimensional motion mode, and a power spectrum analysis is performed. In the frequency band near the main frequency of the heartbeat, the spectral energy proportion of each motion mode and the coherence with the microvascular signal are calculated, and the energy proportion and coherence results are fused to generate the physiological motion matching degree. When the physiological motion matching degree exceeds the preset threshold, it is determined to be consistent with the physiological motion. Secondly, when the high-frequency singularity of the silicone artifact is detected using the enhanced texture feature map, the enhanced texture feature map can be multi-scaled wavelet decomposition to extract the local singularity points of the nonlinear response in the high-frequency sub-band. Based on the time-frequency distribution characteristics of the singularity points, the frequency domain fingerprint template of the silicone material is constructed, the time-frequency ridge difference between the time-frequency distribution characteristics and the frequency domain fingerprint template is calculated, and the kurtosis and skewness indicators of the difference in the spatial domain are statistically analyzed to generate the material abnormality, which characterizes the degree of deviation of the artifact material from the reflection characteristics of the biological tissue. Finally, when calculating the difference in wavelet domain energy distribution between the near-infrared image sequence and the visible light image sequence, the wavelet transform method can be used to decompose the image sequence and extract the phase information of the high-frequency sub-band. By calculating the phase consistency of the high-frequency sub-band, it is determined whether there is a noise attack. When the phase consistency is lower than the preset threshold, a liveness alarm is issued; when the phase consistency is greater than or equal to the preset threshold, further judgment is made based on the physiological motion matching and material abnormality. This application introduces a variety of technical means to perform liveness detection from three aspects: physiological motion matching, material abnormality, and time-frequency domain energy distribution. It effectively solves the technical problems of traditional methods in resisting noise attacks and high-simulation counterfeit attacks, and significantly improves the accuracy and robustness of liveness detection.

[0041] Optionally, calculating the temporal frequency consistency of the microvascular pulsation frequency and the deformation trajectory to obtain the physiological motion matching degree includes: Performing time alignment and bandpass filtering on the pulsation timing signal and the deformation trajectory to respectively extract the main frequency features within the heartbeat frequency band; Decomposing the 3D displacement field of the deformation trajectory into low-dimensional motion modes, and performing power spectrum analysis on each mode; In the frequency band near the main frequency of the heartbeat, the spectrum energy proportion of each motion mode and its coherence with the microvascular signal are calculated; The energy proportion and the coherence result are integrated to generate the physiological motion matching degree, wherein when the physiological motion matching degree exceeds a preset threshold, it is determined that the physiological motion is consistent.

[0042] Specifically, the technical features of the present application include time alignment and bandpass filtering of the pulsation timing signal and the deformation trajectory, respectively extracting the main frequency features within the heartbeat frequency band; decomposing the 3D displacement field of the deformation trajectory into low-dimensional motion modes, and performing power spectrum analysis on each mode; calculating the spectral energy proportion of each motion mode and the coherence with the microvascular signal in the frequency band near the main frequency of the heartbeat; fusing the energy proportion and coherence results to generate the physiological motion matching degree. Through these technical features, the problem of consistency between the microvascular pulsation frequency and the time frequency of the deformation trajectory can be solved. Specifically, the main frequency features within the heartbeat frequency band are extracted through time alignment and bandpass filtering, the 3D displacement field of the deformation trajectory is decomposed into low-dimensional motion modes, and power spectrum analysis is performed to calculate the spectral energy proportion of each motion mode and the coherence with the microvascular signal, and finally these results are fused to generate the physiological motion matching degree, thereby judging the consistency of the physiological motion.

[0043] Exemplarily, the steps of time alignment and bandpass filtering of the beat timing signal and the deformation trajectory can be implemented by a high-precision time synchronization device to ensure that the signals are aligned under the same time reference. Bandpass filtering can be implemented by using a digital filter, such as an FIR filter or an IIR filter, to extract the main frequency characteristics within the heartbeat frequency band. The 3D displacement field of the deformation trajectory is decomposed into low-dimensional motion modes, which can be implemented by dimensionality reduction techniques such as principal component analysis (PCA) or independent component analysis (ICA). Power spectrum analysis can be performed by methods such as fast Fourier transform (FFT) or wavelet transform. In the frequency band near the main frequency of the heartbeat, the spectral energy proportion of each motion mode and its coherence with the microvascular signal are calculated, which can be implemented by spectrum analysis tools such as MATLAB or the SciPy library in Python. Fusion of energy proportion and coherence results to generate physiological motion matching can be achieved by weighted average or other fusion algorithms. This application performs time alignment and bandpass filtering on the pulsation timing signal and the deformation trajectory, extracts the main frequency characteristics within the heartbeat frequency band, decomposes the 3D displacement field of the deformation trajectory into low-dimensional motion modes, and performs power spectrum analysis to calculate the spectral energy proportion of each motion mode and the coherence with the microvascular signal. Finally, these results are integrated to generate the physiological motion matching degree, thereby solving the problem of consistency between the microvascular pulsation frequency and the time frequency of the deformation trajectory. Compared with the prior art, this application can more accurately judge the consistency of physiological motion and improve the accuracy and robustness of liveness detection.

[0044] Optionally, the detecting high-frequency singularity of silicone artifacts by using the enhanced texture feature map to obtain the material abnormality comprises: Performing multi-scale wavelet decomposition on the enhanced texture feature map to extract local singular points of nonlinear response in high-frequency sub-bands; Constructing a frequency domain fingerprint template of the silicone material based on the time-frequency distribution characteristics of the singular point, and calculating the time-frequency ridge difference between the time-frequency distribution characteristics and the frequency domain fingerprint template; The kurtosis and skewness indexes of the difference in the spatial domain are counted to generate the material abnormality, wherein the material abnormality represents the degree of deviation between the reflection characteristics of the artifact material and the biological tissue.

[0045] Specifically, the high-frequency singularity of silicone artifacts is detected by using enhanced texture feature maps; the enhanced texture feature maps are decomposed by multi-scale wavelets to extract local singular points of nonlinear response in high-frequency sub-bands; the frequency domain fingerprint template of silicone material is constructed based on the time-frequency distribution characteristics of the singular points, and the time-frequency ridge difference between the time-frequency distribution characteristics and the frequency domain fingerprint template is calculated; the kurtosis and skewness indicators of the difference in the spatial domain are statistically analyzed to generate the material abnormality. These technical features work together to solve the technical problem of detecting the high-frequency singularity of silicone artifacts through enhanced texture feature maps and obtaining the material abnormality. Specifically, by performing multi-scale wavelet decomposition on the enhanced texture feature maps and extracting local singular points of nonlinear response in high-frequency sub-bands, the high-frequency singularity characteristics of silicone artifacts can be effectively captured. Then, based on the time-frequency distribution characteristics of these singular points, the frequency domain fingerprint template of silicone material is constructed, and the time-frequency ridge difference between these features and the template is calculated, further enhancing the detection capability of silicone artifacts. Finally, the material abnormality is generated by using the kurtosis and skewness indicators of the statistical difference in the spatial domain, thereby characterizing the degree of deviation between the reflective characteristics of the artifact material and the biological tissue, thereby improving the accuracy and robustness of liveness detection.

[0046] Exemplarily, the enhanced texture feature map is subjected to multi-scale wavelet decomposition to extract local singular points of nonlinear response in the high-frequency sub-band. The wavelet decomposition method can use discrete wavelet transform (DWT) or continuous wavelet transform (CWT), etc., to capture the details of different frequency components in the image through multi-scale analysis. The frequency domain fingerprint template of the silicone material is constructed based on the time-frequency distribution characteristics of the singular points, and the time-frequency ridge difference between the time-frequency distribution characteristics and the frequency domain fingerprint template is calculated. The frequency domain fingerprint template can be obtained by training with pre-collected silicone artifact sample images, and the time-frequency ridge difference can be measured by calculating the similarity between the time-frequency distribution characteristics and the frequency domain fingerprint template. The kurtosis and skewness indicators of the statistical difference in the spatial domain generate material abnormality. The kurtosis and skewness indicators can be calculated by statistical methods, such as fitting using a Gaussian distribution model, so as to obtain a quantitative indicator of material abnormality. The technical solution of the present application can effectively solve the artifact detection problem existing in the traditional liveness detection method by using the enhanced texture feature map to detect the high-frequency singularity of silicone artifacts. Compared with the prior art, the advantage of this application is that it can more accurately capture the high-frequency singularity characteristics of silicone artifacts through multi-scale wavelet decomposition and time-frequency analysis methods, thereby improving the accuracy and robustness of detection. Furthermore, by generating material abnormality through the kurtosis and skewness indicators of statistical differences in the spatial domain, it can more comprehensively characterize the degree of deviation between the reflective characteristics of artifact materials and biological tissues, thereby enhancing the reliability of liveness detection.

[0047] Optionally, the method further comprises: When the material abnormality is greater than the second threshold, the reflectivity sampling window is reduced from Reduce to ,in , where is the sensitivity adjustment coefficient, is the abnormality of the material, is the second threshold; Perform edge enhancement operation within the reduced window; The reflectivity difference is calculated using the reduced window to obtain a new reflection anomaly map.

[0048] Specifically, when the material abnormality is detected to be greater than a specific threshold, the reflectivity sampling window is reduced and edge enhancement operations are performed within the reduced window to further calculate the reflectivity difference and generate a new reflection anomaly map. These technical features work together to more accurately identify reflection anomalies when a high degree of material abnormality is detected, thereby improving the accuracy of liveness detection. By reducing the reflectivity sampling window, changes in reflective characteristics of local areas can be detected more centrally and noise interference can be reduced. Performing edge enhancement operations within the reduced window can capture edge features more clearly and further improve the accuracy of reflectivity difference calculations. Ultimately, a new reflection anomaly map is generated, which helps to more accurately determine the liveness situation.

[0049] Exemplarily, when the material abnormality is greater than the second threshold, the specific implementation of reducing the reflectivity sampling window may include the following steps: first, according to the detected material abnormality C and the preset second threshold , calculate the adjustment coefficient ,in is the sensitivity adjustment coefficient. Next, the original reflectivity sampling window Reduce to . An edge enhancement operation is performed within the reduced reflectivity sampling window. Specifically, a variety of edge detection algorithms, such as Sobel, Canny, etc., can be used to enhance edge features. Finally, the reflectivity data within the reduced window is used to calculate the reflectivity difference and generate a new reflection anomaly map. The edge enhancement operation can use a gradient-based edge detection algorithm to further enhance the edge features by calculating the image gradient amplitude map. In this way, the change in reflection characteristics can be captured more accurately and the accuracy of reflectivity difference calculation can be improved. Through the above-mentioned technical solution, when the material abnormality is high, the present application can more accurately identify reflection anomalies by reducing the reflectivity sampling window and edge enhancement operations, thereby improving the accuracy of liveness detection. Compared with the prior art, the method of the present application can better cope with high-simulation counterfeit attacks, reduce the missed detection rate, and improve the robustness and reliability of detection.

[0050] Based on the same inventive concept, Figure 3 As shown, the present invention also provides a living face detection system based on near infrared and visible light binocular cameras, and the system also includes: The multimodal module data processing module is used to synchronously collect binocular visible light and near-infrared images and perform cross-modal alignment to generate binocular visible light image sequences and near-infrared image sequences corresponding to the pixel-level space; A physiological feature generation module, used for extracting subcutaneous microvascular pulsation timing signals from the near-infrared image sequence; A physical feature generation module, used to enhance the texture boundary features in the visible light image sequence by combining the pulsation timing signal and the near infrared image sequence to obtain an enhanced texture feature map; A dynamic deformation verification module generates a facial depth map sequence based on the enhanced texture feature map, and analyzes the 3D deformation continuity and physiological movement consistency in the expression action to obtain a deformation continuity analysis result; The living body judgment module is used to judge the living body through time-frequency domain disturbance detection according to the pulsation timing signal, the enhanced texture feature map and the deformation continuity analysis result.

[0051] It should be noted that the electrical connection between the above-mentioned units does not necessarily mean direct connection of the lines, and the indirect connection mode can be applied to the embodiments of the present invention as long as the purpose of the present invention is achieved. The above is only an exemplary embodiment of the present invention and cannot be used to limit the scope of the present invention.

[0052] That is, any equivalent changes and modifications made according to the teachings of the present invention are still within the scope of the present invention. After considering the disclosure of the specification and the truth of practice, those skilled in the art will easily think of other embodiments of the present invention. This application is intended to cover any variation, use or adaptive change of the present invention, which follows the general principles of the present invention and includes common knowledge or customary technical means in the art that are not described in the present invention.

Claims

1. A method for detecting living faces based on near-infrared and visible light binocular cameras, characterized in that: The method comprises: Synchronously collect binocular visible light and near-infrared images and perform cross-modal alignment to generate binocular visible light image sequences and near-infrared image sequences that correspond to each other at the pixel level. Extracting subcutaneous microvascular pulsation timing signals from the near-infrared image sequence; Combining the pulsation timing signal and the near-infrared image sequence, enhancing the texture boundary features in the visible light image sequence to obtain an enhanced texture feature map; Generate a facial depth map sequence based on the enhanced texture feature map, and analyze the 3D deformation continuity and physiological movement consistency in the facial expression to obtain a deformation continuity analysis result; Based on the pulsation timing signal, the enhanced texture feature map, and the deformation continuity analysis result, liveness determination is performed through time-frequency domain disturbance detection.

2. The method for detecting living human faces based on near infrared and visible light binocular cameras according to claim 1, characterized in that: The synchronous acquisition of binocular visible light and near infrared images and cross-modal alignment to generate binocular visible light image sequences and near infrared image sequences corresponding to pixel-level space includes: Synchronously trigger the near-infrared camera and the visible light binocular camera to obtain the facial near-infrared image and binocular visible light image at the same time; Cross-modal image alignment is performed through SURF feature point matching to generate binocular visible light image sequences and near-infrared image sequences that correspond to each other at the pixel level.

3. The method for detecting living human faces based on near infrared and visible light binocular cameras according to claim 1, characterized in that: The step of extracting the subcutaneous microvascular pulsation timing signal from the near-infrared image sequence comprises: Separating subcutaneous scattering components of the near-infrared image sequence to obtain a first physiological characteristic map; Eliminating the specular reflection noise of the first physiological characteristic map based on the Retinex algorithm to obtain a second physiological characteristic map; The optical flow method is used to track the microvascular pulsation signal in the facial area in the second physiological characteristic image, extract the periodic blood flow change characteristics, and obtain the pulsation timing signal.

4. The method for detecting living human faces based on near infrared and visible light binocular cameras according to claim 1, characterized in that: The step of enhancing the texture boundary features in the visible light image sequence by combining the near infrared image sequence to obtain an enhanced texture feature map comprises: Calculating the microvascular pulsation frequency according to the phase difference of the pulsation timing signal, and generating a reflectivity sampling window aligned with the heartbeat cycle to obtain a window parameter; In the time interval defined by the window parameters, the reflectivity difference in the near infrared band is calculated to generate a material reflection anomaly map; Performing edge detection on the visible light image to generate a gradient amplitude map; Perform channel concatenation on the reflection anomaly map and the gradient amplitude map, and input them into a spectrally sensitive convolutional layer to generate a spatial attention weight; Performing high-frequency filtering on the visible light image to obtain an edge texture feature map; The attention weight is multiplied pixel by pixel by the edge texture feature map to generate an enhanced texture feature map.

5. The method for detecting living human faces based on near infrared and visible light binocular cameras according to claim 4, characterized in that: The facial depth map sequence is generated based on the enhanced texture feature map, and the 3D deformation continuity and physiological movement consistency in the expression action are analyzed to obtain the deformation continuity analysis results including: According to the enhanced texture feature map, an initial depth map is obtained based on binocular disparity calculation; Registering the initial depth map with the facial key points of the near infrared image sequence to obtain a high-precision 3D facial displacement field; The facial displacement field is constrained and filtered using a preset biomechanical model to remove deformation vectors that exceed physiological limits and obtain a deformation trajectory.

6. The method for detecting living human faces based on near infrared and visible light binocular cameras according to claim 5, characterized in that: The performing of living body determination by time-frequency domain disturbance detection according to the pulsation timing signal, the enhanced texture feature map, and the deformation continuity analysis result comprises: Calculating the temporal frequency consistency between the pulsation timing signal and the deformation trajectory to obtain a physiological motion matching degree; The enhanced texture feature map is used to detect the high-frequency singularity of the silicone artifact to obtain the material abnormality; Calculating the difference in wavelet domain energy distribution between the near-infrared image sequence and the visible light image sequence, if the phase consistency of the high-frequency sub-band is lower than a preset threshold, determining that there is an anti-noise attack and issuing a live alarm; If the phase consistency of the high-frequency sub-band is greater than or equal to a preset threshold, a living body alarm is triggered when the physiological movement matching degree is less than a preset first threshold and the material abnormality degree is greater than a preset second threshold.

7. The method for detecting living human faces based on near infrared and visible light binocular cameras according to claim 6, characterized in that: The calculating of the temporal frequency consistency between the microvascular pulsation frequency and the deformation trajectory to obtain the physiological motion matching degree comprises: Performing time alignment and bandpass filtering on the pulsation timing signal and the deformation trajectory to respectively extract the main frequency features within the heartbeat frequency band; Decomposing the 3D displacement field of the deformation trajectory into low-dimensional motion modes, and performing power spectrum analysis on each mode; In the frequency band near the main frequency of the heartbeat, the spectrum energy proportion of each motion mode and its coherence with the microvascular signal are calculated; The energy proportion and the coherence result are integrated to generate the physiological motion matching degree, wherein when the physiological motion matching degree exceeds a preset threshold, it is determined that the physiological motion is consistent.

8. The method for detecting living human faces based on near infrared and visible light binocular cameras according to claim 6, characterized in that: The method of using the enhanced texture feature map to detect the high-frequency singularity of the silicone artifact and obtain the material abnormality includes: Performing multi-scale wavelet decomposition on the enhanced texture feature map to extract local singular points of nonlinear response in high-frequency sub-bands; Constructing a frequency domain fingerprint template of the silicone material based on the time-frequency distribution characteristics of the singular point, and calculating the time-frequency ridge difference between the time-frequency distribution characteristics and the frequency domain fingerprint template; The kurtosis and skewness indexes of the difference in the spatial domain are counted to generate the material abnormality, wherein the material abnormality represents the degree of deviation between the reflection characteristics of the artifact material and the biological tissue.

9. The method for detecting living human faces based on near infrared and visible light binocular cameras according to claim 6, characterized in that: The method further comprises: When the material abnormality is greater than the second threshold, the reflectivity sampling window is reduced from Reduce to ,in , where is the sensitivity adjustment coefficient, is the abnormality of the material, is the second threshold; Perform edge enhancement operation within the reduced window; The reflectivity difference is calculated using the reduced window to obtain a new reflection anomaly map.

10. A living face detection system based on near infrared and visible light binocular cameras, characterized in that: The system comprises: The multimodal module data processing module is used to synchronously collect binocular visible light and near-infrared images and perform cross-modal alignment to generate binocular visible light image sequences and near-infrared image sequences corresponding to the pixel-level space; A physiological feature generation module, used for extracting subcutaneous microvascular pulsation timing signals from the near-infrared image sequence; A physical feature generation module, used to enhance the texture boundary features in the visible light image sequence by combining the pulsation timing signal and the near infrared image sequence to obtain an enhanced texture feature map; A dynamic deformation verification module generates a facial depth map sequence based on the enhanced texture feature map, and analyzes the 3D deformation continuity and physiological movement consistency in the expression action to obtain a deformation continuity analysis result; The living body judgment module is used to judge the living body through time-frequency domain disturbance detection according to the pulsation timing signal, the enhanced texture feature map and the deformation continuity analysis result.

Images