A multi-feature fusion face living body detection method and device and a storage medium
By employing a multi-feature fusion-based face liveness detection method, which combines blinking motion, head posture, and multi-model anti-spoofing discrimination, the problem of insufficient anti-spoofing capability, poor detection stability, and incomplete state management in existing technologies is solved, achieving higher detection accuracy and robustness.
Patent Information
- Application Number
- CN202611250284.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-18
- Publication Date
- 2026-11-06
AI Technical Summary
Existing face liveness detection solutions have shortcomings in anti-spoofing capabilities, detection stability, multi-condition joint verification, and session-level state management. They cannot effectively distinguish between real people and photos/video attacks, and are prone to misjudgment and state interference in online services.
A multi-feature fusion face liveness detection method is adopted, which combines blink action detection, head posture change detection, and multi-model anti-spoofing discrimination with a session-level state management and temporal fusion framework. It utilizes eye aspect ratio, three-state machine, Euler angle change and inference results of multiple anti-spoofing models to perform multi-level verification and result fusion.
It improves the anti-counterfeiting capabilities, detection stability, and accuracy of liveness detection, reduces the success rate of attacks, enhances the robustness and scalability of the system, adapts to a wider range of forged sample types, and reduces the risk of overfitting from a single model.
Smart Images
Figure CN123052833A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of software, and in particular to a method, apparatus and storage medium for multi-feature fusion face liveness detection. Background Technology
[0002] Facial recognition has been widely used in real-name authentication scenarios such as remote account opening, mobile payment, access control, and online government services. However, relying solely on facial comparison cannot distinguish between a real person being present and attacks using photos, screen captures, video playback, or even simple masks. Therefore, liveness detection has become an indispensable part of identity authentication systems.
[0003] Currently, there are three main types of face liveness detection solutions used in the industry. The first type is static feature detection based on a single frame image, which analyzes texture, reflection, sharpness, or signs of tampering to determine if it is a real face. This method is simple to implement, but it is particularly sensitive to changes in lighting, camera quality, image compression, and user posture. A misjudgment of a single image can directly determine the final result, making its stability significantly insufficient. The second type is action-challenge detection, which typically requires the user to cooperate by performing actions such as blinking, shaking their head, or opening their mouth to confirm that the captured object is a real person cooperating in real time. Compared to purely static methods, this interactive requirement can block some simple photo attacks, but if it relies on only one action (such as only detecting blinking), an attacker can bypass it with a high-definition video or screen playback containing blinking. The third type is detection based on multi-frame temporal analysis, which uses the dynamic changes between consecutive image frames to improve the robustness of the judgment. Although it is an improvement over single-frame solutions, vulnerabilities still exist against carefully crafted dynamic attacks when there are no explicit action constraints.
[0004] The closest existing technologies to this invention can mostly be categorized as single-frame liveness detection or single blink detection schemes. These schemes reveal four prominent problems in practical applications.
[0005] First, the detection dimension is limited. Relying solely on single-frame texture information or examining only a blinking action cannot simultaneously cover photo printing attacks, screen replay attacks, and dynamic forgery attacks using video, resulting in an insufficient attack surface.
[0006] Secondly, the results are not stable enough. The single-frame model is very sensitive to factors such as image blurring, head rotation, sudden changes in lighting, and occlusion. Occasional false positives and false negatives will directly reduce the overall pass rate and security, resulting in a poor user experience.
[0007] Third, there is a lack of multi-condition joint verification. Existing solutions typically do not integrate blinking behavior, head posture changes, and liveness scores from consecutive frames into a single framework for fusion judgment. Once a single condition is breached, the entire liveness detection fails, resulting in very low security redundancy.
[0008] Fourth, there is a lack of engineered session-level state management. In scenarios such as online interfaces, batch processing, and streaming detection, a large number of requests from different users are mixed together. If an independent state is not maintained for each detection session, the states of different tasks are prone to interference, leading to misjudgments and affecting the accuracy and scalability of the system.
[0009] The root cause of these shortcomings lies in the fact that most existing technologies only perform liveness detection from a single dimension. They lack a unified design that integrates action interaction verification, posture change verification, deep model anti-spoofing results, and multi-frame statistical information. Furthermore, they fail to establish a complete session state management and result fusion mechanism for real-world deployment environments such as online services. Therefore, the industry needs a liveness detection solution that can simultaneously achieve real-time performance, accuracy, robustness, and engineering deployability. Summary of the Invention
[0010] To address the aforementioned technical problems, this invention provides a multi-feature fusion method for face liveness detection, comprising the following steps:
[0011] Receive a detection request, which includes a session identifier, a frame identifier, and image data; decode the image data to obtain the current image frame.
[0012] For each of the session identifiers, an independent session-level detection state instance is created, and blink detection state, head posture detection state, and liveness detection timing state are maintained in the session-level detection state instance.
[0013] Extract key eye points from the current image frame and calculate the eye aspect ratio. Based on the eye aspect ratio, use a three-state machine mechanism including an initial state, an open-eye confirmation state, and a closed-eye confirmation state to determine whether the blinking action has been completed, and update the blink detection state.
[0014] Extract facial key points from the current image frame, solve the rotation vector based on the correspondence between the facial key points and the preset three-dimensional face reference points, convert the rotation vector into Euler angles, compare the change of the Euler angles between the current image frame and the previous frame to determine whether a head rotation action has occurred, and update the head pose detection state.
[0015] The face region in the current image frame is located and cropped. Multiple anti-spoofing models are invoked to infer the cropped face region. The prediction results of the multiple anti-spoofing models are integrated to obtain a single-frame classification label and a single-frame confidence score. When the single-frame classification label indicates a real face and the single-frame confidence score is greater than or equal to a set score threshold, the current image frame is marked as a real live frame. Otherwise, it is marked as a non-real live frame, and the marking result of the current image frame is written into a sliding window of fixed length.
[0016] After the sliding window reaches a preset length, the proportion of real live frames in the sliding window is counted. When the proportion is greater than or equal to a set pass rate threshold, it is determined that the live condition is met and the live detection timing state is updated.
[0017] Obtain a preset detection mode, and perform logical combination judgment on the updated blink detection state, head posture detection state and liveness detection timing state according to the preset detection mode. When all necessary detection items associated with the preset detection mode meet the passing conditions, output the liveness detection passing result.
[0018] Secondly, the present invention also includes a multi-feature fusion face liveness detection device, comprising:
[0019] The request receiving and decoding module is used to receive a detection request, which includes a session identifier, a frame identifier, and image data, and to decode the image data to obtain the current image frame.
[0020] The session management module is used to create an independent session-level detection state instance for each session identifier, and maintain blink detection state, head posture detection state and liveness detection timing state in the session-level detection state instance.
[0021] The blink detection module is used to extract key eye points in the current image frame and calculate the eye aspect ratio. Based on the eye aspect ratio, a three-state machine mechanism including an initial state, an open-eye confirmation state, and a closed-eye confirmation state is used to determine whether a blinking action has been completed and to update the blink detection state.
[0022] The head pose detection module is used to extract facial key points in the current image frame, solve the rotation vector based on the correspondence between the facial key points and the preset three-dimensional face reference points, convert the rotation vector into Euler angles, compare the change of the Euler angles between the current image frame and the previous frame to determine whether a head rotation action has occurred, and update the head pose detection state.
[0023] The liveness detection and timing module is used to locate and crop the face region in the current image frame, call multiple anti-spoofing models to perform inference on the cropped face region, and integrate the prediction results of the multiple anti-spoofing models to obtain a single-frame classification label and a single-frame confidence score. When the single-frame classification label indicates a real face and the single-frame confidence score is greater than or equal to a set score threshold, the current image frame is marked as a real liveness frame; otherwise, it is marked as a non-real liveness frame, and the marking result of the current image frame is written into a sliding window of fixed length. After the sliding window reaches a preset length, the proportion of real liveness frames in the sliding window is counted. When the proportion is greater than or equal to a set pass rate threshold, the liveness condition is determined to be met, and the liveness detection timing state is updated.
[0024] The fusion judgment module is used to obtain a preset detection mode, and to perform logical combination judgment on the updated blink detection state, head posture detection state and liveness detection timing state according to the preset detection mode. When all the necessary detection items associated with the preset detection mode meet the passing conditions, the liveness detection passing result is output.
[0025] Thirdly, the present invention also includes an electronic device, characterized in that it includes a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, it implements the aforementioned multi-feature fusion face liveness detection method.
[0026] Fourthly, the present invention also includes a computer-readable storage medium, characterized in that it stores a computer program thereon, which, when executed by a processor, implements the aforementioned multi-feature fusion face liveness detection method.
[0027] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0028] This solution improves the actual performance of liveness detection from multiple levels by incorporating blink detection, head pose change detection, and anti-spoofing discrimination through multi-model integration into a session-level state management and temporal fusion framework.
[0029] In terms of anti-counterfeiting capabilities, because it simultaneously verifies blinking behavior, head turning movements, and liveness scores based on depth models, rather than relying solely on a single information source, it is difficult for static photos, screen playback, or some dynamic video forgery attacks to meet the standards in all three dimensions at the same time, significantly reducing the success rate of attacks.
[0030] In terms of detection stability, a fixed-length sliding window is introduced to statistically analyze the liveness results of consecutive frames. Only when the proportion of real live frames in the window reaches a preset threshold is the result considered as passed. In this way, even if individual frames are misjudged due to brief blurring, side profiles, or strong light, the entire session will not be overturned directly, effectively overcoming the problem of large fluctuations in single-frame detection.
[0031] In terms of the reliability of action verification, blink detection adopts a three-state machine of initial-open-eye confirmation-closed-eye confirmation. By observing the changing trend of the aspect ratio of the eyes in multiple consecutive frames, it can accurately identify a complete blink action and filter out false triggers caused by unconscious eyelid tremors and instantaneous noise. Head posture detection confirms whether the user has made a significant head turning action by comparing the changes in posture such as yaw angle between consecutive frames, adding a second explicit action defense line for liveness detection.
[0032] In terms of multi-model generalization capability, the liveness detection process integrates the inference results of multiple anti-spoofing models and sets a confidence threshold for single-frame output. The complementary advantages between different models can adapt to a wider range of forged sample types, reduce the risk of overfitting of a single model, and make the overall detection accuracy and robustness better than a single model solution. Attached Figure Description
[0033] Figure 1 This is the overall flowchart;
[0034] Figure 2 This is a structural block diagram of the human face liveness detection system of the present invention;
[0035] Figure 3 This is a schematic diagram of the session-level detection state management of the present invention;
[0036] Figure 4 This is a flowchart of the blink detection state machine of the present invention;
[0037] Figure 5 This is a flowchart of the head posture detection process of the present invention;
[0038] Figure 6 This is a flowchart of the liveness detection model integration and sliding window decision process of the present invention;
[0039] Figure 7 This is a flowchart of the multi-mode fusion determination process of the present invention;
[0040] Figure 8 This is a schematic diagram of the real-time streaming detection interaction of the present invention. Detailed Implementation
[0041] The terms first, second, third, fourth, etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms include or have, and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0042] like Figures 1-8 A multi-feature fusion face liveness detection method includes the following steps:
[0043] Receive a detection request, which includes a session identifier, a frame identifier, and image data; decode the image data to obtain the current image frame.
[0044] The server receives a detection request from the client, which includes at least a session identifier (session_id), a frame identifier (frame_id), and image data. The session identifier distinguishes independent detection sessions from different users; the frame identifier indicates the temporal order of image frames within the same session; the image data can be a Base64-encoded string or a binary image stream. The server decodes the image data to obtain the current image frame to be processed, for example, decoding it into an image matrix in RGB color space.
[0045] For each of the session identifiers, an independent session-level detection state instance is created, and blink detection state, head posture detection state, and liveness detection timing state are maintained in the session-level detection state instance.
[0046] For each unique session identifier, the server creates an independent session-level detection state instance in memory. This instance maintains the intermediate states and results of various detection tasks within that session, thus supporting complete isolation of multi-user detection states in high-concurrency scenarios. Specifically, this session-level detection state instance maintains the following state information:
[0047] (1) Blink detection state: includes at least the current state value of the three-state machine (such as the initial state, the open-eye confirmation state, and the closed-eye confirmation state represented by integers), the closed-eye count used to count the number of consecutive closed-eye frames, and a Boolean flag indicating whether at least one complete blink action has been completed.
[0048] (2) Head posture detection status: including at least the Euler angles of the previous frame (such as yaw, pitch, roll), instantaneous markers of whether head rotation is detected, and completion markers of whether at least one head rotation has been completed.
[0049] (3) Liveness detection timing status: including at least a fixed-length sliding window buffer queue for storing the marking results of the last few frames, the proportion of real live frames in the current window, and a completion mark indicating whether the liveness pass condition is met.
[0050] Extract key eye points from the current image frame and calculate the eye aspect ratio. Based on the eye aspect ratio, use a three-state machine mechanism including an initial state, an open-eye confirmation state, and a closed-eye confirmation state to determine whether the blinking action has been completed, and update the blink detection state.
[0051] Extract facial key points from the current image frame, solve the rotation vector based on the correspondence between the facial key points and the preset three-dimensional face reference points, convert the rotation vector into Euler angles, compare the change of the Euler angles between the current image frame and the previous frame to determine whether a head rotation action has occurred, and update the head pose detection state.
[0052] The face region in the current image frame is located and cropped. Multiple anti-spoofing models are invoked to infer the cropped face region. The prediction results of the multiple anti-spoofing models are integrated to obtain a single-frame classification label and a single-frame confidence score. When the single-frame classification label indicates a real face and the single-frame confidence score is greater than or equal to a set score threshold, the current image frame is marked as a real live frame. Otherwise, it is marked as a non-real live frame, and the marking result of the current image frame is written into a sliding window of fixed length.
[0053] After the sliding window reaches a preset length, the proportion of real live frames in the sliding window is counted. When the proportion is greater than or equal to a set pass rate threshold, it is determined that the live condition is met and the live detection timing state is updated.
[0054] Obtain a preset detection mode, and perform logical combination judgment on the updated blink detection state, head posture detection state and liveness detection timing state according to the preset detection mode. When all necessary detection items associated with the preset detection mode meet the passing conditions, output the liveness detection passing result.
[0055] As a preferred technical solution, the method of determining whether a blinking action has been completed based on the eye aspect ratio using a three-state machine mechanism that includes an initial state, an open-eye confirmation state, and a closed-eye confirmation state includes:
[0056] When the three-state machine is in the initial state and the eye aspect ratio is greater than the eye-opening threshold, the three-state machine is switched to the eye-opening confirmation state.
[0057] When the tri-state machine is in the open-eye confirmation state and the eye aspect ratio is below the closed-eye threshold for multiple consecutive frames, the tri-state machine is switched to the closed-eye confirmation state.
[0058] When the three-state machine is in the closed-eye confirmation state and detects that the eye aspect ratio is greater than the open-eye threshold again, it is determined that a complete blinking action has been completed.
[0059] As a preferred technical solution, the extraction of facial key points in the current image frame includes extracting two-dimensional feature points of the nose tip, chin, outer corner of the left eye, outer corner of the right eye, left corner of the mouth, and right corner of the mouth;
[0060] The step of comparing the change in Euler angles between the current image frame and the previous frame to determine whether a head rotation action has occurred includes:
[0061] Obtain the yaw angle from the Euler angles;
[0062] Calculate the change in yaw angle between the current image frame and the previous frame;
[0063] When the change in yaw angle exceeds the set attitude change threshold, a head rotation action is determined to have occurred.
[0064] The preset detection modes include at least one of the following modes: blink detection plus liveness detection mode, blink detection plus head posture detection mode, head posture detection plus liveness detection mode, liveness detection only mode, and blink detection plus head posture detection plus liveness detection mode.
[0065] As a preferred technical solution, when the received detection request is a batch image processing request containing multiple images to be detected, the multiple images to be detected are processed sequentially one by one; the method further includes:
[0066] After outputting the liveness detection result, images with a single-frame confidence score greater than or equal to a preset value are selected from the candidate images to be detected to form a candidate image set;
[0067] The image with the largest aspect ratio of the eye is selected from the candidate image set and output as the preferred open-eye image.
[0068] As a preferred technical solution, receiving the detection request includes: continuously receiving a detection request containing consecutive video frames using a real-time communication method, performing detection on each of the consecutive video frames and returning the real-time detection result of the current frame;
[0069] When the liveness detection result is output, a stop signal command is sent to the client to instruct the client to stop sending subsequent video frames.
[0070] As a preferred technical solution, the states maintained in the session-level detection state instance specifically include:
[0071] The blink detection state includes the current state of the state machine, the number of closed eyes, and a marker indicating whether at least one blink has been completed.
[0072] The head posture detection status includes the posture angle of the previous frame, a marker indicating whether a head rotation action was detected, and a marker indicating whether at least one head rotation action was completed.
[0073] The liveness detection timing status includes a cache queue of multi-frame marking results, the current proportion of real liveness frames, and a marking indicating whether the liveness detection pass condition has been met.
[0074] The present invention also relates to a face liveness detection device with multi-feature fusion, comprising:
[0075] The request receiving and decoding module is used to receive a detection request, which includes a session identifier, a frame identifier, and image data, and to decode the image data to obtain the current image frame.
[0076] The session management module is used to create an independent session-level detection state instance for each session identifier, and maintain blink detection state, head posture detection state and liveness detection timing state in the session-level detection state instance.
[0077] The blink detection module is used to extract key eye points in the current image frame and calculate the eye aspect ratio. Based on the eye aspect ratio, a three-state machine mechanism including an initial state, an open-eye confirmation state, and a closed-eye confirmation state is used to determine whether a blinking action has been completed and to update the blink detection state.
[0078] The head pose detection module is used to extract facial key points in the current image frame, solve the rotation vector based on the correspondence between the facial key points and the preset three-dimensional face reference points, convert the rotation vector into Euler angles, compare the change of the Euler angles between the current image frame and the previous frame to determine whether a head rotation action has occurred, and update the head pose detection state.
[0079] The liveness detection and timing module is used to locate and crop the face region in the current image frame, call multiple anti-spoofing models to perform inference on the cropped face region, and integrate the prediction results of the multiple anti-spoofing models to obtain a single-frame classification label and a single-frame confidence score. When the single-frame classification label indicates a real face and the single-frame confidence score is greater than or equal to a set score threshold, the current image frame is marked as a real liveness frame; otherwise, it is marked as a non-real liveness frame, and the marking result of the current image frame is written into a sliding window of fixed length. After the sliding window reaches a preset length, the proportion of real liveness frames in the sliding window is counted. When the proportion is greater than or equal to a set pass rate threshold, the liveness condition is determined to be met, and the liveness detection timing state is updated.
[0080] The fusion judgment module is used to obtain a preset detection mode, and to perform logical combination judgment on the updated blink detection state, head posture detection state and liveness detection timing state according to the preset detection mode. When all the necessary detection items associated with the preset detection mode meet the passing conditions, the liveness detection passing result is output.
[0081] The present invention will now be described in detail with reference to preferred embodiments.
[0082] Implementation Method 1: Single-Frame Liveness Detection Based on Interface Calls
[0083] The client sends a detection request to the server, which includes a session identifier, a frame identifier, and image data. Upon receiving the request, the server decodes the image data to obtain the current image frame.
[0084] The server first checks if a corresponding detection instance already exists based on the session identifier. If it does not exist, a new detection instance is initialized. This detection instance includes:
[0085] Blink detector;
[0086] Head pose detector;
[0087] Liveness detector;
[0088] Historical state cache.
[0089] Subsequently, the server performs blink detection, head pose detection, and liveness detection on the current image frame.
[0090] 1. Blink detection process
[0091] After converting the image frames to grayscale, the face region is detected, key points for the left and right eyes are extracted, and the eye elasticity (EAR) of the left and right eyes is calculated. If the current state is the initial state and the EAR is greater than the eye-opening threshold, the system enters the eye-opening confirmation state. If the current state is the eye-opening confirmation state and the EAR is lower than the eye-closing threshold for several consecutive frames, the system enters the eye-closing confirmation state. If the current state is the eye-closing confirmation state and the EAR is again higher than the eye-opening threshold, a complete blink is determined to have been completed.
[0092] 2. Head posture detection process
[0093] Feature points such as the tip of the nose, chin, corners of the eyes, and corners of the mouth are extracted from facial keypoints and mapped to preset 3D facial reference points. A pose solving algorithm is used to calculate rotation vectors, which are then converted to Euler angles. By comparing the change in yaw angle between the current frame and the previous frame, if it exceeds a preset threshold, it is determined that the head has rotated.
[0094] 3. Liveness detection process
[0095] First, the bounding boxes of faces in the current image are detected, and the face regions are cropped. Then, multiple anti-spoofing models are invoked to perform inference on the cropped image, and the predictions of all models are accumulated or integrated to obtain the classification label and score of the current frame.
[0096] If the current frame is classified as a real face and the score is greater than or equal to a preset threshold, the frame is recorded as a real live frame; otherwise, it is recorded as a non-real live frame. The frame result is then written into a fixed-length sliding window. When the sliding window is full, the proportion of real frames is calculated. If the proportion of real frames reaches the pass rate threshold, the session is marked as live.
[0097] 4. Fusion Determination Process
[0098] The system performs fusion according to a preset detection mode. For example, in the blink detection + liveness detection mode, the system outputs "pass" only if the session has completed at least one blink and the timing liveness result is "pass"; otherwise, it outputs "fail".
[0099] Implementation Method Two: Batch Image Processing Implementation Method
[0100] The client uploads multiple images at once. The server parses and processes the images sequentially, generating a detection result for each image. If an image during processing causes the current session to meet the fusion pass criteria, the task is marked as passed.
[0101] After the task is completed, the system filters the target image from all candidate images. Preferably, images with a liveness score greater than or equal to a preset value are selected first, and then the EAR values of these images are compared. The image with the largest EAR value is selected as the preferred open-eye image for output.
[0102] If no optimal image that meets the criteria is found, the last image passed will be selected as the output.
[0103] Implementation Method 3: Real-time Streaming Processing Implementation Method
[0104] The client continuously sends video frames via WebSocket. Upon receiving each frame, the server immediately performs image decoding, detection processing, and result transmission. The system can be configured with a stop-on-the-fly strategy based on business parameters. When the detection passes, the client stops sending subsequent video frames, thereby reducing system computational overhead and user waiting time.
[0105] Implementation Method 4: Result Retention and Auditing
[0106] In a preferred embodiment, the system overlays detection information onto the image, including:
[0107] EAR value;
[0108] blinking state;
[0109] Living state;
[0110] Subsequently, a directory structure is created based on date, hour, and session identifier, and the tagged images are saved. This implementation method facilitates subsequent issue tracking, audit documentation, and model optimization.
[0111] In a preferred embodiment, each threshold can be set according to business needs, including but not limited to:
[0112] Eye-opening threshold;
[0113] Eye-closing threshold;
[0114] Threshold for the number of consecutive frames with eyes closed;
[0115] Single-frame liveness score threshold;
[0116] Sliding window length;
[0117] Vital pass rate threshold;
[0118] Threshold for head posture change.
[0119] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A multi-feature fusion face liveness detection method, characterized in that, Includes the following steps: Receive a detection request, which includes a session identifier, a frame identifier, and image data; decode the image data to obtain the current image frame. For each of the session identifiers, an independent session-level detection state instance is created, and blink detection state, head posture detection state, and liveness detection timing state are maintained in the session-level detection state instance. Extract key eye points from the current image frame and calculate the eye aspect ratio. Based on the eye aspect ratio, use a three-state machine mechanism including an initial state, an open-eye confirmation state, and a closed-eye confirmation state to determine whether the blinking action has been completed, and update the blink detection state. Extract facial key points from the current image frame, solve the rotation vector based on the correspondence between the facial key points and the preset three-dimensional face reference points, convert the rotation vector into Euler angles, compare the change of the Euler angles between the current image frame and the previous frame to determine whether a head rotation action has occurred, and update the head pose detection state. The face region in the current image frame is located and cropped. Multiple anti-spoofing models are invoked to infer the cropped face region. The prediction results of the multiple anti-spoofing models are integrated to obtain the single-frame classification label and single-frame confidence score. When the single-frame classification label indicates a real human face and the single-frame confidence score is greater than or equal to a set score threshold, the current image frame is marked as a real live frame; otherwise, it is marked as a non-real live frame, and the marking result of the current image frame is written into a sliding window of fixed length. After the sliding window reaches a preset length, the proportion of real live frames in the sliding window is counted. When the proportion is greater than or equal to a set pass rate threshold, it is determined that the live condition is met and the live detection timing state is updated. Obtain a preset detection mode, and perform logical combination judgment on the updated blink detection state, head posture detection state and liveness detection timing state according to the preset detection mode. When all necessary detection items associated with the preset detection mode meet the passing conditions, output the liveness detection passing result.
2. The face liveness detection method based on multi-feature fusion according to claim 1, characterized in that, The method for determining whether a blinking action has been completed based on the eye aspect ratio using a three-state machine mechanism that includes an initial state, an open-eye confirmation state, and a closed-eye confirmation state includes: When the three-state machine is in the initial state and the eye aspect ratio is greater than the eye-opening threshold, the three-state machine is switched to the eye-opening confirmation state. When the tri-state machine is in the open-eye confirmation state and the eye aspect ratio is below the closed-eye threshold for multiple consecutive frames, the tri-state machine is switched to the closed-eye confirmation state. When the three-state machine is in the closed-eye confirmation state and detects that the eye aspect ratio is greater than the open-eye threshold again, it is determined that a complete blinking action has been completed.
3. The face liveness detection method based on multi-feature fusion according to claim 1, characterized in that, The extraction of facial key points in the current image frame includes extracting two-dimensional feature points of the tip of the nose, chin, outer corner of the left eye, outer corner of the right eye, left corner of the mouth, and right corner of the mouth. The step of comparing the change in Euler angles between the current image frame and the previous frame to determine whether a head rotation action has occurred includes: Obtain the yaw angle from the Euler angles; Calculate the change in yaw angle between the current image frame and the previous frame; When the change in yaw angle exceeds the set attitude change threshold, a head rotation action is determined to have occurred.
4. The face liveness detection method based on multi-feature fusion according to claim 1, characterized in that, The preset detection modes include at least one of the following modes: blink detection plus liveness detection mode, blink detection plus head posture detection mode, head posture detection plus liveness detection mode, liveness detection only mode, and blink detection plus head posture detection plus liveness detection mode.
5. The face liveness detection method based on multi-feature fusion according to claim 1, characterized in that, When the received detection request is a batch image processing request containing multiple images to be detected, the multiple images to be detected are processed sequentially one by one; the method further includes: After outputting the liveness detection result, images with a single-frame confidence score greater than or equal to a preset value are selected from the candidate images to be detected to form a candidate image set; The image with the largest aspect ratio of the eye is selected from the candidate image set and output as the preferred open-eye image.
6. The face liveness detection method based on multi-feature fusion according to claim 1, characterized in that, The receiving of detection requests includes: continuously receiving detection requests containing consecutive video frames using a real-time communication method, performing detection frame by frame on the consecutive video frames and returning the real-time detection result of the current frame; When the liveness detection result is output, a stop signal command is sent to the client to instruct the client to stop sending subsequent video frames.
7. The face liveness detection method based on multi-feature fusion according to claim 1, characterized in that, The states maintained in the session-level detection state instance specifically include: The blink detection state includes the current state of the state machine, the number of closed eyes, and a marker indicating whether at least one blink has been completed. The head posture detection status includes the posture angle of the previous frame, a marker indicating whether a head rotation action was detected, and a marker indicating whether at least one head rotation action was completed. The liveness detection timing status includes a cache queue of multi-frame marking results, the current proportion of real liveness frames, and a marking indicating whether the liveness detection pass condition has been met.
8. A face liveness detection device with multi-feature fusion, characterized in that, include: The request receiving and decoding module is used to receive a detection request, which includes a session identifier, a frame identifier, and image data, and to decode the image data to obtain the current image frame. The session management module is used to create an independent session-level detection state instance for each session identifier, and maintain blink detection state, head posture detection state and liveness detection timing state in the session-level detection state instance. The blink detection module is used to extract key eye points in the current image frame and calculate the eye aspect ratio. Based on the eye aspect ratio, a three-state machine mechanism including an initial state, an open-eye confirmation state, and a closed-eye confirmation state is used to determine whether a blinking action has been completed and to update the blink detection state. The head pose detection module is used to extract facial key points in the current image frame, solve the rotation vector based on the correspondence between the facial key points and the preset three-dimensional face reference points, convert the rotation vector into Euler angles, compare the change of the Euler angles between the current image frame and the previous frame to determine whether a head rotation action has occurred, and update the head pose detection state. The liveness model verification and temporal module is used to locate the face region in the current image frame and crop it, call multiple anti-spoofing models to perform inference on the cropped face region respectively, and integrate the prediction results of multiple anti-spoofing models to obtain a single-frame classification label and a single-frame confidence score. When the single-frame classification label indicates a real human face and the single-frame confidence score is greater than or equal to a set score threshold, the current image frame is marked as a real live frame; otherwise, it is marked as a non-real live frame, and the marking result of the current image frame is written into a sliding window of fixed length. After the sliding window reaches a preset length, the proportion of real live frames in the sliding window is counted. When the proportion is greater than or equal to a set pass rate threshold, it is determined that the live condition is met and the live detection timing state is updated. The fusion judgment module is used to obtain a preset detection mode, and to perform logical combination judgment on the updated blink detection state, head posture detection state and liveness detection timing state according to the preset detection mode. When all the necessary detection items associated with the preset detection mode meet the passing conditions, the liveness detection passing result is output.
9. An electronic device, characterized in that, It includes a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, it implements the multi-feature fusion face liveness detection method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, It stores a computer program, which, when executed by a processor, implements the multi-feature fusion face liveness detection method as described in any one of claims 1 to 7.