A method and system for implementing liveness detection

Abstract

The present invention discloses a method and system for implementing liveness detection. The method can first perform static detection such as whether the user is a real person and whether the face is in the detection area when the user performs identity verification. The present invention uses multi-feature fusion technology in deep learning to extract features at different layers by designing a neural network structure, construct a feature pyramid structure to fuse feature maps of different scales, and use the final feature layer to perform static detection. After the user passes the static detection, the present invention can use voice prompts to guide the user to perform a series of dynamic facial movements and collect the coordinates of the user's facial feature points when the user performs the movements in real time. The dynamic trajectory determined by the facial feature point coordinates is used to verify whether the user's real live operation is based on the dynamic trajectory. The present invention can not only effectively identify fraud and impersonation and improve the accuracy of recognition and detection, but also save a lot of detection costs and improve the efficiency of recognition and detection by combining dynamic detection and static detection.

Description

Technical Field

[0001] The present invention relates to the technical field of identity authentication, and in particular to a method and system for liveness detection. Background Art

[0002] With the development of technology and communications, a variety of smart devices and smart applications have become increasingly common, becoming an integral part of people's lives and bringing significant convenience. Since these devices and applications record a wide range of user data, to protect the user's personal privacy and security, user identification and authentication are required during use to prevent data theft.

[0003] One of the commonly used identity recognition methods is face recognition, which involves the user pre-entering their facial features. When triggered, the smart terminal then obtains the current user's facial features and compares the current facial features with the previously entered facial features to determine the identity of the triggering user.

[0004] The currently commonly used face recognition methods have the following technical problems: since facial features are extracted when the user is static, criminals can extract features by stealing the user's photos or avatars to impersonate the user and steal data, increasing the risk of user data leakage and lowering security and confidentiality. Summary of the Invention

[0005] The present invention proposes a liveness detection method and system, which respectively perform static and liveness detection on the detected user, and combine static and dynamic detection to determine the user's identity, thereby avoiding impersonation, improving the accuracy of identity recognition, and protecting the user's privacy and security.

[0006] A first aspect of an embodiment of the present invention provides a liveness detection method, the method comprising:

[0007] When the user triggers identity verification, a preset neural network is used to detect the user to prevent duplication, and a first set of video frames of the user's face is collected in real time. The first set of video frames includes multiple continuous video frames of the user when the user is static.

[0008] Static detection of facial feature information contained in the first video frame set, wherein the facial feature information includes the number of faces and the areas where the faces are located;

[0009] When the facial feature information satisfies a preset static feature condition, a preset dynamic instruction voice is played, and a second video frame set is captured when the user performs a dynamic action corresponding to the dynamic instruction voice, and a facial dynamic feature set is extracted from the second video frame set, wherein the second video frame set includes multiple consecutive video frames when the user performs the dynamic action, and the facial dynamic feature set includes multiple facial feature key points;

[0010] Dynamic detection is performed based on the facial dynamic feature set to determine the user identity.

[0011] In a possible implementation of the first aspect, performing dynamic detection based on the facial dynamic feature set to determine the user identity includes:

[0012] Normalizing the plurality of facial feature key points to obtain a plurality of normalized key points;

[0013] Searching for a feature coordinate point corresponding to each of the normalized key points in each video frame of the second video frame set to obtain a plurality of feature coordinate points;

[0014] Determining a dynamic trajectory of the user when performing a dynamic action using the multiple feature coordinate points;

[0015] If the dynamic trajectory is the same as the indication trajectory corresponding to the preset dynamic indication voice, it is determined that the user identity is passed, wherein the indication trajectory includes: eye trajectory and mouth trajectory;

[0016] If the dynamic trajectory is different from the indication trajectory corresponding to the preset dynamic indication voice, it is determined that the user identity is not passed.

[0017] In a possible implementation of the first aspect, determining a dynamic trajectory of the user when performing a dynamic action by using the multiple feature coordinate points includes:

[0018] Convert each of the feature coordinate points into a template coordinate point corresponding to a preset 3D face template to obtain a plurality of template coordinate points;

[0019] Calculating a rotation vector of the user's face relative to a preset 3D face template using the multiple template coordinate points;

[0020] The rotation vector is used to calculate an angle change value when the user performs a dynamic action, and the user's action trajectory is determined based on the angle change value.

[0021] In a possible implementation of the first aspect, calculating the angle change value when the user performs a dynamic action by using the rotation vector includes:

[0022] Calculating the L2 norm of the rotation vector to obtain the vector angle;

[0023] Calculate the radian angle corresponding to the vector angle by trigonometric functions to obtain the angle value with direction;

[0024] Count multiple angle values ​​when the user performs the dynamic action corresponding to the dynamic instruction voice, and calculate the angle change value using the multiple angle values.

[0025] In a possible implementation of the first aspect, the statically detecting facial feature information contained in the first set of video frames includes:

[0026] Using a preset face detection model to identify and count the number and location of faces in each video frame in the first video frame set, to obtain the number of faces and the areas where the faces are located;

[0027] When the number of faces is equal to a preset number and the area where the faces are located overlaps with the preset area, it is determined that the preset static feature condition is met;

[0028] Otherwise, it is determined that the preset static characteristic condition is not satisfied.

[0029] In a possible implementation of the first aspect, the preset face detection model is specifically:

[0030] The image model is obtained by training the image model using training image samples, wherein the image model is constructed according to the YOLO target detection algorithm.

[0031] In a possible implementation of the first aspect, the anti-copy detection is specifically:

[0032] Collecting a user image that triggers identity authentication, and performing binary classification accelerated reasoning on the user image using a preset neural network to determine whether the user portrait in the user image is a real portrait;

[0033] If the user portrait in the user image is a real portrait, executing the step of capturing a first set of video frames about the user's face in real time;

[0034] If the user portrait in the user image is not a real portrait, then stop triggering the identity verification operation;

[0035] The preset neural network is a dual-branch network structure designed by using the Depthwise SeparableConvolution structural unit of the Mobilenet structure and extracting the sixth layer output and cascading it with the branch structure branch. The training parameters are set and distributed training is performed using PyTorch. The dual-branch network structure is divided into a main branch structure and a secondary branch structure. The main branch structure is used to predict the category of the network, and the secondary branch structure is used to supervise the training of the network.

[0036] The training parameter setting is specifically as follows: loading the weight file of the preset neural network into the memory and setting it as a static variable, and when the user calls the preset neural network, using the weight file of the preset neural network as a shared weight matrix;

[0037] The trunk branch structure uses the Softmax function as the loss function to predict the category, and the supervision branch structure uses the mean square error MSE;

[0038] The loss of the trunk-branch structure is:

[0039]

[0040] The loss of the supervision branch is:

[0041]

[0042] The total loss of the network is:

[0043]

[0044] In a possible implementation of the first aspect, after the step of statically detecting facial feature information included in the first set of video frames, the method further includes:

[0045] Expand the length and width of the area where the face is located in equal proportion to obtain an expanded area;

[0046] extracting boundary information from the expanded area;

[0047] The boundary information is used to construct a preset area.

[0048] In a possible implementation of the first aspect, after the step of performing dynamic detection based on the facial dynamic feature set to determine the user identity, the method further includes:

[0049] The quantization method is used to perform linear transformation and data mapping on the original data of the preset neural network to obtain the adjusted data;

[0050] The adjustment data is used to fine-tune and optimize the preset neural network.

[0051] A second aspect of an embodiment of the present invention provides a living body detection system, the system comprising:

[0052] A static capture module, configured to, when a user triggers identity verification, use a preset neural network to detect the user and prevent duplication, and to capture a first set of video frames of the user's face in real time, the first set of video frames comprising multiple consecutive video frames of the user in a static state;

[0053] a static detection module, configured to statically detect facial feature information contained in the first set of video frames, wherein the facial feature information includes the number of faces and the areas where the faces are located;

[0054] a dynamic acquisition module, configured to play a preset dynamic instruction voice when the facial feature information satisfies a preset static feature condition, and to capture a second set of video frames when the user performs a dynamic action corresponding to the dynamic instruction voice, and to extract a facial dynamic feature set from the second set of video frames, wherein the second set of video frames includes multiple consecutive video frames of the user performing the dynamic action, and the facial dynamic feature set includes multiple facial feature key points;

[0055] The dynamic detection module is used to perform dynamic detection based on the facial dynamic feature set to determine the user's identity.

[0056] Compared with the existing technology, the liveness detection method and system provided by the embodiments of the present invention have the following beneficial effects: the present invention can perform anti-copying detection, static detection and dynamic detection on the user respectively when the user triggers identity authentication, and determine the user's identity when all the detections are passed, so as to improve the accuracy of identity recognition, and avoid the situation where criminals can impersonate the user by stealing the user's photos or avatars for feature extraction, thereby reducing the risk of user data leakage or theft, increasing security and confidentiality, and protecting the user's privacy. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 1 is a flow chart of a liveness detection method provided by one embodiment of the present invention;

[0058] Figure 2 This is an operational flow chart of a liveness detection method provided by one embodiment of the present invention;

[0059] Figure 3 It is a structural diagram of a living body detection system provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0060] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0061] Currently commonly used face recognition methods have the following technical problems: Since facial features are extracted when the user is static, criminals can extract features by stealing the user's photos or avatars to impersonate the user and steal data, increasing the risk of user data leakage and lowering security and confidentiality.

[0062] In order to solve the above problems, a liveness detection method provided in an embodiment of the present application will be introduced and explained in detail through the following specific embodiments.

[0063] Reference Figure 1 , which shows a flow chart of a liveness detection method provided by an embodiment of the present invention.

[0064] As an example, the liveness detection method may include:

[0065] S11. When the user triggers identity authentication, a preset neural network is used to detect the user to prevent duplication, and a first set of video frames of the user's face is collected in real time. The first set of video frames includes multiple continuous video frames of the user when he is static.

[0066] In one embodiment, the user can trigger identity authentication in any scenario, for example, it can be identity authentication for user login, or identity authentication for fee payment.

[0067] After determining that the user has triggered this identity authentication, the user can be subjected to anti-copying detection in real time. The anti-copying detection is a detection operation used to determine whether the user uses a face photo for identity authentication. The anti-copying detection can prevent someone from obtaining user information by stealing user photos.

[0068] When the anti-copying detection passes, the facial image of the user who triggered the identity verification can be collected in real time. In one application mode, multiple frames of facial images of the user in a static state can be collected in real time and continuously to construct the first video frame set.

[0069] In one embodiment, the anti-copying detection is specifically:

[0070] Collecting a user image that triggers identity authentication, and performing binary classification accelerated reasoning on the user image using a preset neural network to determine whether the user portrait in the user image is a real portrait;

[0071] If the user portrait in the user image is a real portrait, executing the step of capturing a first set of video frames about the user's face in real time;

[0072] If the user portrait in the user image is not a real portrait, then stop triggering the identity verification operation;

[0073] The preset neural network is a dual-branch network structure designed by using the Depthwise SeparableConvolution structural unit of the Mobilenet structure and extracting the sixth layer output and cascading it with the branch structure branch. The training parameters are set and distributed training is performed using PyTorch. The dual-branch network structure is divided into a main branch structure and a secondary branch structure. The main branch structure is used to predict the category of the network, and the secondary branch structure is used to supervise the training of the network.

[0074] The training parameter setting is specifically as follows: loading the weight file of the preset neural network into the memory and setting it as a static variable, and when the user calls the preset neural network, using the weight file of the preset neural network as a shared weight matrix;

[0075] The trunk branch structure uses the Softmax function as the loss function to predict the category, and the supervision branch structure uses the mean square error MSE;

[0076] The loss of the trunk-branch structure is:

[0077]

[0078] The loss of the supervision branch is:

[0079]

[0080] The total loss of the network is:

[0081]

[0082] By preventing fake photo detection, the user's face image can be classified into two categories, thereby distinguishing whether the detected face image is a real portrait or a fake portrait, avoiding impersonation.

[0083] In one embodiment, the bottom layer can use Vulkan to perform matrix inference calculations on each model. Since the image is also a set of matrices in all forward inference process calculations, Vulkan can accelerate the calculations and improve the efficiency of matrix inference calculations.

[0084] Since the calculated results are different each time because the image is changing, the ultimate goal of using Vulkan is to accelerate the calculation, and the corresponding output is the network's calculation results.

[0085] S12. Statically detect facial feature information contained in the first video frame set, wherein the facial feature information includes the number of faces and the areas where the faces are located.

[0086] After confirming that the person who triggered the verification operation is a genuine user, a video stream of the user's face can be captured in real time and then divided into multiple video frames to obtain a first video frame set. Since each frame in the video stream contains an image of the user's face, after the division, the user in each frame is in a static state. Therefore, the first video frame set contains multiple consecutive video frames that all contain images of the user's face when the user is in a static state.

[0087] Then, the facial features of the user in each video frame can be detected to extract the facial feature information.

[0088] In one embodiment, the facial feature information may include the number of faces and the areas where the faces are located.

[0089] Specifically, the number of faces may be the number of faces contained in each video frame. The area where the face is located is the specific area position of the face image in each video frame.

[0090] During use, if there are multiple users triggering verification, impersonation or false triggering may occur easily. If the area where the user is located when triggering is not in the detection area, it may also be a false trigger by the user. To avoid false triggering, in an optional embodiment, step S12 may include the following sub-steps:

[0091] Sub-step S121: using a preset face detection model to identify and count the number and position of faces in each video frame in the first video frame set, to obtain the number of faces and the area where the faces are located.

[0092] Specifically, the number of faces in all video frames in the first video frame set can be calculated, and then the total number of faces can be divided by the video frame data to obtain the number of faces. Alternatively, the face positions in all video frames in the first video frame set can be calculated, and then all face positions can be averaged to obtain the face area.

[0093] In one embodiment, the preset face detection model is specifically:

[0094] The image model is obtained by training the image model using training image samples, wherein the image model is constructed according to the YOLO target detection algorithm.

[0095] In one embodiment, multiple different or identical face detection models can be trained, each face detection model corresponding to the detection of one user, and multiple face detection models can share a set of matrix parameters for corresponding detection when using multiple face detection models for concurrent inference by setting global shared model parameters, thereby saving memory overhead.

[0096] When in use, through global sharing, when multiple users create or trigger verification, the network weight matrix is ​​directly copied from the memory. Among them, the network weight matrix can be the shared weight matrix mentioned above.

[0097] When using the inference framework to load the model, the weight file of the loaded model is loaded into memory and set as a static variable. When multiple users create, this set of shared weight matrices is directly loaded from memory, which can effectively avoid multiple loading issues, significantly reduce IO operations, and increase memory usage.

[0098] Since the network model is essentially a matrix weight file, this shared weight matrix is ​​the model file obtained through the above network training. This model is globally unique because multiple threads actually use this set of weights to calculate the network output results. Therefore, there is no need to load it multiple times, and you only need to set it to static global uniqueness.

[0099] In an optional embodiment, a model quantization technique can be used to optimize the face detection model. Specifically, the model quantization technique uses the face detection model and key points to extract network optimization elements, and then optimizes each element.

[0100] Sub-step S122: When the number of faces is equal to a preset number value and the area where the faces are located overlaps with the preset area, it is determined that the preset static feature condition is met.

[0101] Sub-step S123: Otherwise, it is determined that the preset static characteristic condition is not satisfied.

[0102] By using the two conditions of the number of faces and the area where the faces are located, it is possible to determine whether the user is falsely triggered, thereby improving the accuracy and security of identity authentication, and also reducing the increase in verification workload and power consumption due to false triggering.

[0103] During verification, different users have different heights and weights, and their distances from the camera vary. For example, a user who is close to the camera may have their face outside the preset area. The area outside the preset area is difficult to detect, which can reduce detection accuracy.

[0104] In order to avoid the above situation, as much facial feature information of the user as possible is collected. In one embodiment, after step S12, the method may further include:

[0105] S21. Expand the length and width of the area where the face is located in equal proportion to obtain an expanded area.

[0106] Specifically, after determining the area where the face is located, the length and width of the area where the face is located may be respectively widened according to a preset expansion ratio.

[0107] For example, if the face area is 5*5 cm, the length and width of the face area can be widened at an expansion ratio of 1.1, so that the face area is expanded to 5.5*5.5 cm, thereby obtaining an expanded area.

[0108] It should be noted that the preset expansion ratio can be adjusted according to actual needs, and this application is not limited thereto.

[0109] S22: Extract boundary information from the expanded area.

[0110] After the expanded area is obtained, the length and width of the expanded area may be obtained respectively to obtain boundary information of the expanded area.

[0111] S23: Construct a preset area using the boundary information.

[0112] Then, the preset area may be reconstructed according to the boundary information, so that the preset area detected later is expanded.

[0113] The preset area is larger and contains more information, so the preset area can be cropped to obtain a new image containing the boundary area. It should be noted that all operations will not change the original input image. Some operations such as cropping will copy a new image of the area for operation.

[0114] By expanding the preset area, more information about the user's facial image and the surrounding information of the facial image can be effectively extracted to improve the detection accuracy.

[0115] S13. When the facial feature information meets the preset static feature conditions, play the preset dynamic instruction voice, and collect a second video frame set when the user performs the dynamic action corresponding to the dynamic instruction voice, and extract the facial dynamic feature set from the second video frame set, wherein the second video frame set includes multiple continuous video frames when the user performs the dynamic action, and the facial dynamic feature set includes multiple facial feature key points.

[0116] In one embodiment, after determining that the preset static feature conditions are met, a preset dynamic instruction voice can be played for the user to listen to. The dynamic instruction voice is used to prompt the user to perform a corresponding dynamic action, such as raising a hand, drawing a stroke, blinking, or shaking the head.

[0117] At the same time, when the user performs a dynamic action, the camera is called to capture a video stream in real time. The video stream contains multiple continuous video frames, and the second video frame set is constructed by the multiple video frames.

[0118] Moreover, since the camera captures the user's facial area, each video frame contains multiple facial feature key points when the user performs dynamic actions, and multiple facial feature key points can constitute a facial dynamic feature set.

[0119] S14: Perform dynamic detection based on the facial dynamic feature set to determine the user identity.

[0120] Finally, the facial dynamic feature set can be used to perform corresponding dynamic detection to determine whether the user meets the detection requirements and to determine the user's true identity.

[0121] Since different users may perform different dynamic actions, a user may not be able to perform a dynamic action, or may make an error. In order to accurately determine whether the user identity can pass the dynamic detection, in one embodiment, step S14 may include the following sub-steps:

[0122] Sub-step S141 : normalizing the plurality of facial feature key points to obtain a plurality of normalized key points.

[0123] Specifically, each facial feature key point may be normalized separately, thereby obtaining multiple normalized key points.

[0124] Sub-step S142: searching for a feature coordinate point corresponding to each normalized key point in each video frame of the second video frame set to obtain a plurality of feature coordinate points.

[0125] Specifically, each normalized key point can be projected into the world coordinate system to obtain the feature coordinate point corresponding to each normalized key point.

[0126] Sub-step S143: using the plurality of characteristic coordinate points to determine the dynamic trajectory of the user when performing the dynamic action.

[0127] The deflection angle information of the user's face is calculated using multiple feature coordinate points, and then combined with multiple deflection angle information generated by continuous video frames of the second video frame set to determine the dynamic trajectory of the user when performing dynamic actions.

[0128] In order to accurately determine the dynamic trajectory of the user when performing a dynamic action, in one embodiment, sub-step S143 may include the following sub-steps:

[0129] Sub-step S1431: converting each of the feature coordinate points into a template coordinate point corresponding to a preset 3D face template to obtain a plurality of template coordinate points.

[0130] Sub-step S1432: Calculate the rotation vector of the user's face relative to the preset 3D face template using the multiple template coordinate points.

[0131] Sub-step S1433: Calculate the angle change value when the user performs a dynamic action using the rotation vector, and determine the user's action trajectory based on the angle change value.

[0132] In order to further improve the accuracy of calculating the angle change value, in one embodiment, sub-step S1433 may include the following sub-steps:

[0133] Sub-step S14331: Calculate the L2 norm of the rotation vector to obtain the vector angle.

[0134] Sub-step S14332: Calculate the radian angle corresponding to the vector angle using trigonometric functions to obtain a directional angle value.

[0135] Sub-step S14333: Count multiple angle values ​​when the user performs the dynamic action corresponding to the dynamic instruction voice, and calculate the angle change value using the multiple angle values.

[0136] In an optional application, the feature coordinate points can also be converted into three-dimensional space information points, and then the radian angle between the vectors can be obtained by inverse trigonometric function calculation, and then the radian angle is converted to obtain the deflection angle information.

[0137] Finally, the differences between the multiple deflection angle information generated by the continuous video frames of the second video frame set are calculated to obtain the angle change value, and finally the user's motion trajectory is edited using the angle change value.

[0138] Sub-step S144: If the dynamic trajectory is the same as the indication trajectory corresponding to the preset dynamic indication voice, it is determined that the user identity is passed, and the indication trajectory includes: eye trajectory and mouth trajectory.

[0139] Sub-step S145: If the dynamic trajectory is different from the indication trajectory corresponding to the preset dynamic indication voice, it is determined that the user identity is not passed.

[0140] Among them, the eye trajectory can be obtained by using the coordinates of two points at the corners of the eyes to calculate the horizontal distance of the eyes to obtain Width, and then using the four key points of the upper and lower eyelids to calculate the vertical distance of the eyes to obtain Height. After obtaining Width and Height, the horizontal and vertical coordinate ratio of the eyes can be calculated EyeRatio = Width / Height to obtain the eye trajectory.

[0141] Similarly, the mouth trajectory can also be obtained by first using the coordinates of the two points of the mouth corners to calculate the horizontal distance of the mouth to obtain Width, and then using the four key points of the upper and lower lips to calculate the vertical distance of the mouth Height. After obtaining Width and Height, the horizontal and vertical coordinate ratio of the mouth can be calculated MouthRatio = Width / Height to obtain the mouth trajectory.

[0142] When comparing two trajectories, the changing trajectories of the horizontal and vertical coordinate ratios (the horizontal and vertical coordinate ratios of the eyes or the horizontal and vertical coordinate ratios of the mouth) between different video frames can be analyzed to determine whether the two trajectories are the same, so as to determine whether the user has performed corresponding blinking or mouth opening actions.

[0143] In an optional embodiment, the indication track may also be a shaking track.

[0144] The user's motion trajectory can also be calculated using the above method and compared with the head shaking trajectory. The head shaking trajectory can also be calculated using the above method for calculating the eye trajectory and mouth trajectory. The width is calculated using the horizontal coordinates of the head, and the height is calculated using the coordinates of the top of the head and chin.

[0145] During the comparison process, the user may move from the initial state to the target position and then return to the initial state. This action can be regarded as completing the closed-loop operation of the dynamic action. For example: for the shaking head operation, the initial state position can be set to meet the conditions. When the conditions are met, the initial angle value of the current state facing the camera is recorded. Then, when the shaking head operation is detected, the change in the head distance from the initial state value is calculated in real time. When this change value reaches the preset value, the intermediate state value can be recorded. If the angle value at this time is still increasing in the same direction, this angle value can be updated. When the angle direction begins to change continuously, it can be known that the user has started to shake his head in the opposite direction. When returning to the initial state, it can also be determined that the entire closed-loop operation of the dynamic action is completed, and this detection is completed.

[0146] In order to make the detection and recognition of each model more accurate, in one embodiment, the method may further include:

[0147] S15. Use a quantization method to perform linear transformation and data mapping on the original data of the preset neural network to obtain adjusted data.

[0148] S16. Fine-tune and optimize the preset neural network using the adjustment data.

[0149] In one embodiment, quantization can be performed using 8-bit or 16-bit integers instead of floating-point numbers. This method attempts to use fixed-point dot products instead of floating-point dot products, significantly reducing the computational overhead of neural networks on devices without hard floating-point support. This method can also reduce the model's memory and storage footprint.

[0150] Specifically, you can use the framework's built-in quantization tool, which essentially converts floating-point numbers into fixed-point numbers. For example, if the original model parameters are [0.000502, 0.400001, 0.551100], the new model parameters after quantization are [2, 200, 240]. For details, please refer to the quantization paper.

[0151] Since the quantification method is a normalization method similar to deviation standardization, it performs a linear transformation on the original data so that the result is mapped within a certain range, thereby obtaining the required adjustment data.

[0152] In this embodiment, fine-tuning optimization may be to use adjustment data to make subtle adjustments to the weights of the model that needs to be quantized, in the same way as training the model.

[0153] The specific steps are to use the tools provided by the framework to use 5000 training image samples, input the original model network file and the network weight file into the ncnn2table tool, and enter the output directory and file name.

[0154] Fine-tune the accuracy of the quantized model to obtain the final model for deploying inference calculations.

[0155] Optionally, the optimization of the above face detection model may also be performed using the above method.

[0156] Reference Figure 2 , shows an operational flow chart of a liveness detection method provided by an embodiment of the present invention.

[0157] Specifically, when it is determined that the user triggers the verification operation, a video stream about the user is used and continuous video frames are obtained from the video stream. Then, the video frames are subjected to the first stage of static detection to determine whether the video stream contains real facial graphics. If not, it is considered that the false trigger has timed out. If it is contained, it is determined whether it is a single facial image and the area where the facial image is located. When all requirements are met, the user can be instructed to perform dynamic actions, and the action trajectory of the user performing dynamic actions can be obtained, and the action trajectory can be analyzed. When the trajectory analysis determines that the user has completed the operation, the verification is determined to be successful. Otherwise, the verification is determined to be unsuccessful.

[0158] In this embodiment, an embodiment of the present invention provides a liveness detection method, the beneficial effect of which is that the present invention can perform anti-copying detection, static detection and dynamic detection on the user respectively when the user triggers identity authentication, and determine the user's identity when all detections are passed, so as to improve the accuracy of identity recognition, and avoid the situation where criminals can impersonate the user by stealing the user's photos or avatars for feature extraction, thereby reducing the risk of user data leakage or theft, increasing security and confidentiality, and protecting the user's privacy.

[0159] The embodiment of the present invention also provides a living body detection system, see Figure 3 , which shows a structural diagram of a living body detection system provided by an embodiment of the present invention.

[0160] As an example, the living body detection system may include:

[0161] A static capture module 301 is configured to, when a user triggers identity verification, use a preset neural network to detect the user's face and capture a first set of video frames of the user's face in real time. The first set of video frames includes multiple consecutive video frames of the user in a static state.

[0162] A static detection module 302 is configured to statically detect facial feature information contained in the first set of video frames, wherein the facial feature information includes the number of faces and the areas where the faces are located;

[0163] A dynamic acquisition module 303 is configured to, when the facial feature information satisfies a preset static feature condition, play a preset dynamic instruction voice, and capture a second set of video frames when the user performs a dynamic action corresponding to the dynamic instruction voice, and extract a facial dynamic feature set from the second set of video frames, wherein the second set of video frames includes multiple consecutive video frames of the user performing the dynamic action, and the facial dynamic feature set includes multiple facial feature key points;

[0164] The dynamic detection module 304 is configured to perform dynamic detection based on the facial dynamic feature set to determine the user's identity.

[0165] Optionally, the dynamic detection module is further configured to:

[0166] Normalizing the plurality of facial feature key points to obtain a plurality of normalized key points;

[0167] Searching for a feature coordinate point corresponding to each of the normalized key points in each video frame of the second video frame set to obtain a plurality of feature coordinate points;

[0168] Determining a dynamic trajectory of the user when performing a dynamic action using the multiple feature coordinate points;

[0169] If the dynamic trajectory is the same as the indication trajectory corresponding to the preset dynamic indication voice, it is determined that the user identity is passed, wherein the indication trajectory includes: eye trajectory and mouth trajectory;

[0170] If the dynamic trajectory is different from the indication trajectory corresponding to the preset dynamic indication voice, it is determined that the user identity is not passed.

[0171] Optionally, the dynamic detection module is further configured to:

[0172] Convert each of the feature coordinate points into a template coordinate point corresponding to a preset 3D face template to obtain a plurality of template coordinate points;

[0173] Calculating a rotation vector of the user's face relative to a preset 3D face template using the multiple template coordinate points;

[0174] The rotation vector is used to calculate an angle change value when the user performs a dynamic action, and the user's action trajectory is determined based on the angle change value.

[0175] Optionally, the dynamic detection module is further configured to:

[0176] Calculating the L2 norm of the rotation vector to obtain the vector angle;

[0177] Calculate the radian angle corresponding to the vector angle by trigonometric functions to obtain the angle value with direction;

[0178] Count multiple angle values ​​when the user performs the dynamic action corresponding to the dynamic instruction voice, and calculate the angle change value using the multiple angle values.

[0179] Optionally, the static detection module is further configured to:

[0180] Using a preset face detection model to identify and count the number and location of faces in each video frame in the first video frame set, to obtain the number of faces and the areas where the faces are located;

[0181] When the number of faces is equal to a preset number and the area where the faces are located overlaps with the preset area, it is determined that the preset static feature condition is met;

[0182] Otherwise, it is determined that the preset static characteristic condition is not satisfied.

[0183] Optionally, the preset face detection model is specifically:

[0184] The image model is obtained by training the image model using training image samples, wherein the image model is constructed according to the YOLO target detection algorithm.

[0185] Optionally, the anti-copying detection is specifically:

[0186] Collecting a user image that triggers identity authentication, and performing binary classification accelerated reasoning on the user image using a preset neural network to determine whether the user portrait in the user image is a real portrait;

[0187] If the user portrait in the user image is a real portrait, executing the step of capturing a first set of video frames about the user's face in real time;

[0188] If the user portrait in the user image is not a real portrait, then stop triggering the identity verification operation;

[0189] The preset neural network is a dual-branch network structure designed by using the Depthwise SeparableConvolution structural unit of the Mobilenet structure and extracting the sixth layer output and cascading it with the branch structure branch. The training parameters are set and distributed training is performed using PyTorch. The dual-branch network structure is divided into a main branch structure and a secondary branch structure. The main branch structure is used to predict the category of the network, and the secondary branch structure is used to supervise the training of the network.

[0190] The training parameter setting is specifically as follows: loading the weight file of the preset neural network into the memory and setting it as a static variable, and when the user calls the preset neural network, using the weight file of the preset neural network as a shared weight matrix;

[0191] The trunk branch structure uses the Softmax function as the loss function to predict the category, and the supervision branch structure uses the mean square error MSE;

[0192] The loss of the trunk-branch structure is:

[0193]

[0194] The loss of the supervision branch is:

[0195]

[0196] The total loss of the network is:

[0197]

[0198] Optionally, after the step of statically detecting facial feature information contained in the first set of video frames, the system further includes:

[0199] An expansion module, configured to expand the length and width of the area where the face is located in equal proportion to obtain an expanded area;

[0200] a boundary module, configured to extract boundary information from the expanded area;

[0201] The region construction module is used to construct a preset region using the boundary information.

[0202] Optionally, the system further comprises:

[0203] The data adjustment module is used to perform linear transformation and data mapping on the original data of the preset neural network using a quantization method to obtain adjusted data;

[0204] The fine-tuning and optimization module is used to use the adjustment data to fine-tune and optimize the preset neural network.

[0205] Furthermore, an embodiment of the present application also provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the liveness detection method as described in the above embodiment is implemented.

[0206] Furthermore, an embodiment of the present application also provides a computer-readable storage medium, which stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the liveness detection method as described in the above embodiment.

[0207] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for detecting a living body, characterized in that: The method comprises: When the user triggers identity verification, a preset neural network is used to detect the user to prevent duplication, and a first set of video frames of the user's face is collected in real time. The first set of video frames includes multiple continuous video frames of the user when the user is static. Static detection of facial feature information contained in the first video frame set, wherein the facial feature information includes the number of faces and the areas where the faces are located; When the facial feature information satisfies a preset static feature condition, a preset dynamic instruction voice is played, and a second video frame set is captured when the user performs a dynamic action corresponding to the dynamic instruction voice, and a facial dynamic feature set is extracted from the second video frame set, wherein the second video frame set includes multiple consecutive video frames when the user performs the dynamic action, and the facial dynamic feature set includes multiple facial feature key points; Performing dynamic detection based on the facial dynamic feature set to determine the user's identity; The performing dynamic detection based on the facial dynamic feature set to determine the user identity includes: Normalizing the plurality of facial feature key points to obtain a plurality of normalized key points; Searching for a feature coordinate point corresponding to each of the normalized key points in each video frame of the second video frame set to obtain a plurality of feature coordinate points; Determining a dynamic trajectory of the user when performing a dynamic action using the multiple feature coordinate points; If the dynamic trajectory is the same as the indication trajectory corresponding to the preset dynamic indication voice, it is determined that the user identity is passed, wherein the indication trajectory includes: eye trajectory and mouth trajectory; If the dynamic trajectory is different from the indication trajectory corresponding to the preset dynamic indication voice, it is determined that the user identity is not passed; The determining of the dynamic trajectory of the user when performing the dynamic action by using the multiple feature coordinate points includes: Convert each of the feature coordinate points into a template coordinate point corresponding to a preset 3D face template to obtain a plurality of template coordinate points; Calculating a rotation vector of the user's face relative to a preset 3D face template using the multiple template coordinate points; The rotation vector is used to calculate an angle change value when the user performs a dynamic action, and the user's action trajectory is determined based on the angle change value.

2. The living body detection method according to claim 1, characterized in that The calculating the angle change value when the user performs a dynamic action by using the rotation vector includes: Calculating the L2 norm of the rotation vector to obtain the vector angle; Calculate the radian angle corresponding to the vector angle by trigonometric functions to obtain the angle value with direction; Count multiple angle values ​​when the user performs the dynamic action corresponding to the dynamic instruction voice, and calculate the angle change value using the multiple angle values.

3. The living body detection method according to claim 1, characterized in that: The static detection of facial feature information contained in the first video frame set includes: Using a preset face detection model to identify and count the number and location of faces in each video frame in the first video frame set, to obtain the number of faces and the areas where the faces are located; When the number of faces is equal to a preset number and the area where the faces are located overlaps with the preset area, it is determined that the preset static feature condition is met; Otherwise, it is determined that the preset static characteristic condition is not satisfied.

4. The method for liveness detection according to claim 3, wherein: The preset face detection model is specifically: The image model is obtained by training the image model using training image samples, wherein the image model is constructed according to the YOLO target detection algorithm.

5. The living body detection method according to claim 1, wherein: The anti-copy detection is specifically as follows: Collecting a user image that triggers identity authentication, and performing binary classification accelerated reasoning on the user image using a preset neural network to determine whether the user portrait in the user image is a real portrait; If the user portrait in the user image is a real portrait, executing the step of capturing a first set of video frames about the user's face in real time; If the user portrait in the user image is not a real portrait, then stop triggering the identity verification operation; The preset neural network is a dual-branch network structure designed by using the Depthwise SeparableConvolution structural unit of the Mobilenet structure and extracting the sixth layer output and cascading it with the branch structure branch. The training parameters are set and distributed training is performed using PyTorch. The dual-branch network structure is divided into a main branch structure and a secondary branch structure. The main branch structure is used to predict the category of the network, and the secondary branch structure is used to supervise the training of the network. The training parameter setting is specifically as follows: loading the weight file of the preset neural network into the memory and setting it as a static variable, and when the user calls the preset neural network, using the weight file of the preset neural network as a shared weight matrix; The trunk branch structure uses the Softmax function as the loss function to predict the category, and the supervision branch structure uses the mean square error MSE; The loss of the trunk-branch structure is: The loss of the supervision branch is: The total loss of the network is:

6. The method for liveness detection according to claim 3, wherein: After the step of statically detecting facial feature information contained in the first set of video frames, the method further includes: Expand the length and width of the area where the face is located in equal proportion to obtain an expanded area; extracting boundary information from the expanded area; The boundary information is used to construct a preset area.

7. The liveness detection method according to any one of claims 1 to 6, characterized in that: After the step of performing dynamic detection based on the facial dynamic feature set to determine the user identity, the method further includes: The quantization method is used to perform linear transformation and data mapping on the original data of the preset neural network to obtain the adjusted data; The adjustment data is used to fine-tune and optimize the preset neural network.

8. A living body detection system, characterized in that: The system comprises: A static capture module, configured to, when a user triggers identity verification, use a preset neural network to detect the user and prevent duplication, and to capture a first set of video frames of the user's face in real time, the first set of video frames comprising multiple consecutive video frames of the user in a static state; a static detection module, configured to statically detect facial feature information contained in the first set of video frames, wherein the facial feature information includes the number of faces and the areas where the faces are located; a dynamic acquisition module, configured to play a preset dynamic instruction voice when the facial feature information satisfies a preset static feature condition, and to capture a second set of video frames when the user performs a dynamic action corresponding to the dynamic instruction voice, and to extract a facial dynamic feature set from the second set of video frames, wherein the second set of video frames includes multiple consecutive video frames of the user performing the dynamic action, and the facial dynamic feature set includes multiple facial feature key points; A dynamic detection module, configured to perform dynamic detection based on the facial dynamic feature set to determine the user's identity; The dynamic detection module is also used for: Normalizing the plurality of facial feature key points to obtain a plurality of normalized key points; Searching for a feature coordinate point corresponding to each of the normalized key points in each video frame of the second video frame set to obtain a plurality of feature coordinate points; Determining a dynamic trajectory of the user when performing a dynamic action using the multiple feature coordinate points; If the dynamic trajectory is the same as the indication trajectory corresponding to the preset dynamic indication voice, it is determined that the user identity is passed, wherein the indication trajectory includes: eye trajectory and mouth trajectory; If the dynamic trajectory is different from the indication trajectory corresponding to the preset dynamic indication voice, it is determined that the user identity is not passed; The dynamic detection module is also used for: Convert each of the feature coordinate points into a template coordinate point corresponding to a preset 3D face template to obtain a plurality of template coordinate points; Calculating a rotation vector of the user's face relative to a preset 3D face template using the multiple template coordinate points; The rotation vector is used to calculate an angle change value when the user performs a dynamic action, and the user's action trajectory is determined based on the angle change value.

Images