Liveness detection methods, devices, equipment and storage media

Abstract

This specification provides a liveness detection method, apparatus, device, and storage medium. The method includes: acquiring an authentication request initiated by a target detection object for a transaction; generating authentication interaction information based on the authentication request, wherein the authentication interaction information is used to instruct the target detection object to perform an authentication action; collecting the authentication action of the target detection object to generate an authentication action video; acquiring a reference action video corresponding to the authentication action; and comparing the reference action video with the authentication action video to obtain a liveness detection result of the target detection object; wherein the reference action video is generated based on the registration action collected by the target detection object when registering a transaction.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a method, apparatus, device and storage medium for detecting liveness. Background Technology

[0002] While facial recognition technology brings convenience to people's lives and work, it also introduces additional risks. A typical example is liveness detection, which involves attempts to impersonate a user using photos, mobile phones, or other media. Liveness detection is currently the most threatening security vulnerability facing facial recognition systems. The solution to this problem is usually called liveness detection, which aims to determine whether the acquired biometric features belong to a living, present, and real person.

[0003] In recent years, with the maturity of deep learning-based generative techniques, various attack methods that generate motion videos from photos have posed new challenges to liveness detection algorithms. Therefore, there is an urgent need to propose an effective liveness detection method to address liveness attack incidents. Summary of the Invention

[0004] The main purpose of this specification is to provide a method, apparatus, device, and storage medium for liveness detection, aiming to achieve liveness detection and solve the problem of poor defense against spoofed samples. The technical solution is as follows:

[0005] Firstly, embodiments of this specification provide a method for detecting liveness, including:

[0006] Obtain the authentication request initiated by the target detection object for the transaction, generate authentication interaction information based on the authentication request, and use the authentication interaction information to instruct the target detection object to perform an authentication action.

[0007] The authentication actions of the target detection object are collected to generate an authentication action video;

[0008] Obtain the reference action video corresponding to the authentication action, and compare the reference action video with the authentication action video to obtain the liveness detection result of the target detection object; the reference action video is generated based on the registration action collected when the target detection object registers the transaction.

[0009] Secondly, embodiments of this specification provide a training method for a liveness detection model, including:

[0010] Initialize the initial structural parameters in the action prediction model;

[0011] Acquire biometric sample images of live specimens;

[0012] The biometric sample image is used as input to the action prediction model to obtain the predicted identity label and the predicted preferred registration action;

[0013] The identity tag loss value corresponding to the predicted identity tag and the action prediction loss value corresponding to the predicted preferred registration action are calculated using the action prediction loss function.

[0014] The initial structural parameters are adjusted based on the identity label loss value and the action prediction loss value to obtain the target structural parameters, and the trained action prediction model is generated based on the target structural parameters.

[0015] Thirdly, embodiments of this specification provide a method for training a liveness detection model, including:

[0016] Initialize the initial structural parameters in the simulated motion video generation model;

[0017] Obtain the preferred registration action sample video and the sample candidate registration action tags for live sample objects;

[0018] The preferred registered action sample video and the sample candidate registered action tags are used as inputs to the simulated action generation model to obtain the sample simulated action video;

[0019] The video reconstruction loss value and the constraint loss value corresponding to the sample simulated action video are calculated using the simulated action video generation loss function.

[0020] The initial structural parameters are adjusted based on the video reconstruction loss and the constraint loss to obtain the target structural parameters, and the trained simulated action generation model is generated based on the target structural parameters.

[0021] Fourthly, embodiments of this specification provide a method for training a liveness detection model, including:

[0022] Initialize the initial structural parameters in the action feature comparison model;

[0023] Obtain a video of the authentication action of a live sample object and a simulated video of the authentication action.

[0024] The sample authentication action video and the sample authentication action simulation video are input into the action feature comparison model to obtain the sample authentication action video fusion feature of the sample authentication action video and the sample authentication action simulation video fusion feature of the sample authentication action simulation video.

[0025] The multi-source feature reconstruction loss value and the liveness detection loss value of the sample authentication action video fusion feature and the sample authentication action simulation video fusion feature are calculated using the action feature comparison loss function.

[0026] The initial structural parameters are adjusted based on the multi-source feature reconstruction loss value and the liveness detection loss value to obtain the target structural parameters, and the trained action feature comparison model is generated based on the target structural parameters.

[0027] Fifthly, embodiments of this specification provide a liveness detection device, comprising:

[0028] The authentication interaction module is used to obtain the identity authentication request initiated by the target detection object for the transaction, generate authentication interaction information based on the identity authentication request, and instruct the target detection object to perform an authentication action.

[0029] The acquisition module is used to acquire the authentication actions of the target detection object to generate an authentication action video;

[0030] The detection module is used to acquire a reference action video corresponding to the authentication action, and compare the reference action video with the authentication action video to obtain the liveness detection result of the target detection object; the reference action video is generated based on the registration action collected when the target detection object registers the transaction.

[0031] Sixthly, embodiments of this specification provide a liveness detection device, comprising:

[0032] The first parameter setting module is used to initialize the initial structural parameters in the action prediction model;

[0033] The first acquisition module is used to acquire biometric sample images of live sample objects;

[0034] The prediction module is used to take the biometric sample image as input to the action prediction model to obtain the predicted identity label and the predicted preferred registration action;

[0035] The action prediction loss calculation module is used to calculate the identity tag loss value corresponding to the predicted identity tag and the action prediction loss value corresponding to the predicted preferred registration action using the action prediction loss function.

[0036] The first training module is used to adjust the initial structural parameters based on the identity label loss value and the action prediction loss value to obtain the target structural parameters, and to generate the trained action prediction model based on the target structural parameters.

[0037] Seventhly, embodiments of this specification provide a liveness detection device, comprising:

[0038] The second parameter setting module is used to initialize the initial structural parameters in the simulated motion video generation model.

[0039] The second acquisition module is used to acquire the preferred registration action sample video and the sample alternative registration action tags of the live sample object;

[0040] The simulation module is used to take the preferred registered action sample video and the sample candidate registered action tags as input to the simulated action generation model to obtain the sample simulated action video;

[0041] The simulated motion video generation loss calculation module is used to calculate the video reconstruction loss value and the constraint loss value corresponding to the sample simulated motion video using the simulated motion video generation loss function.

[0042] The second training module is used to adjust the initial structural parameters based on the video reconstruction loss value and the constraint loss to obtain the target structural parameters, and to generate the trained simulated action generation model based on the target structural parameters.

[0043] Eighthly, embodiments of this specification provide a liveness detection device, comprising:

[0044] The third parameter setting module is used to initialize the initial structural parameters in the action feature comparison model.

[0045] The third acquisition module is used to acquire a sample authentication action video of a live sample object and a sample authentication action simulation video of the authentication action;

[0046] The feature fusion module is used to input the sample authentication action video and the sample authentication action simulation video into the action feature comparison model to obtain the sample authentication action video fusion feature of the sample authentication action video and the sample authentication action simulation video fusion feature of the sample authentication action simulation video.

[0047] The multi-source feature reconstruction loss calculation module is used to calculate the multi-source feature reconstruction loss value and the liveness detection loss value of the sample authentication action video fusion feature and the sample authentication action simulation video fusion feature using the action feature comparison loss function;

[0048] The third training module is used to adjust the initial structural parameters based on the multi-source feature reconstruction loss value and the liveness detection loss value to obtain the target structural parameters, and to generate the trained action feature comparison model based on the target structural parameters.

[0049] Ninthly, embodiments of this specification provide an electronic device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the method described above.

[0050] In a tenth aspect, embodiments of this specification provide a storage medium storing a computer program, wherein the liveness detection program, when executed by a processor, implements the steps of the method described above.

[0051] Eleventhly, embodiments of this specification provide a computer program product, including: a computer program that, when executed by a processor of an electronic device, enables the processor to at least implement the methods described in the first to fourth aspects.

[0052] In the embodiments of this specification, by acquiring the authentication request initiated by the target detection object for a transaction, authentication interaction information is generated based on the authentication request. This authentication interaction information instructs the target detection object to perform an authentication action. The authentication action of the target detection object is collected to generate an authentication action video. A reference action video corresponding to the authentication action is acquired, and the reference action video is compared with the authentication action video to obtain the liveness detection result of the target detection object. The reference action video is generated based on the registration action collected by the target detection object when registering a transaction. By obtaining a backup reference video based on the registration action video of the target detection object, and comparing the differences between the current authentication action video of the target detection object and the corresponding backup reference action video during authentication, non-liveness attacks are detected, and video-based attacks can also be detected, resulting in higher accuracy in liveness detection. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 This is a schematic diagram illustrating an example of a liveness detection method provided in the embodiments of this specification;

[0055] Figure 2 This is a schematic flowchart of a liveness detection method provided in the embodiments of this specification;

[0056] Figure 3 This is a schematic diagram of the overall process of a liveness detection method provided in the embodiments of this specification;

[0057] Figure 4 This is a detailed flowchart illustrating a liveness detection method provided in the embodiments of this specification;

[0058] Figure 5 This is a schematic diagram of the registration process of a liveness detection method provided in the embodiments of this specification;

[0059] Figure 6 This is a schematic diagram of the training process of a liveness detection model provided in the embodiments of this specification;

[0060] Figure 7 This is a schematic diagram of the training process of a liveness detection model provided in the embodiments of this specification;

[0061] Figure 8 This is a schematic diagram of the training process of a liveness detection model provided in the embodiments of this specification;

[0062] Figure 9 This is a schematic diagram of the structure of a liveness detection device provided in the embodiments of this specification;

[0063] Figure 10 This is a schematic diagram of the structure of a liveness detection device provided in the embodiments of this specification;

[0064] Figure 11 This is a schematic diagram of the structure of a liveness detection device provided in the embodiments of this specification;

[0065] Figure 12 This is a schematic diagram of the structure of a liveness detection device provided in the embodiments of this specification;

[0066] Figure 13 This is a schematic diagram of the structure of a liveness detection device provided in the embodiments of this specification. Detailed Implementation

[0067] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0068] Furthermore, it should be noted that the acquisition, storage, use, and processing of data involved in the embodiments of this specification, such as face recognition and liveness detection, all comply with the relevant provisions of national laws and regulations.

[0069] The liveness detection device can be a terminal device such as a mobile phone, computer, tablet, smartwatch, or in-vehicle device, or it can be a module in the terminal device used to implement the liveness detection method. The liveness detection device can obtain the identity authentication request initiated by the target detection object for the transaction, generate authentication interaction information based on the identity authentication request, and use the authentication interaction information to instruct the target detection object to perform authentication actions; collect the authentication actions of the target detection object to generate an authentication action video; obtain the reference action video corresponding to the authentication action, and compare the reference action video with the authentication action video to obtain the liveness detection result of the target detection object; the reference action video is generated based on the registration actions collected by the target detection object when registering the transaction.

[0070] Correspondingly, the liveness detection device can also train a liveness detection model. The liveness detection model includes an action prediction model. The liveness detection device can initialize the initial structural parameters in the action prediction model; acquire biometric sample images of live objects; use the biometric sample images as input to the action prediction model to obtain predicted identity labels and predicted preferred registration actions; calculate the identity label loss value corresponding to the predicted identity label and the action prediction loss value corresponding to the predicted preferred registration action using the action prediction loss function; adjust the initial structural parameters based on the identity label loss value and the action prediction loss value to obtain target structural parameters; and generate the trained action prediction model based on the target structural parameters.

[0071] Furthermore, the liveness detection model includes a simulated action video generation model. The liveness detection device can initialize the initial structural parameters in the simulated action video generation model to obtain preferred registered action sample videos and candidate registered action labels for live sample objects; use the preferred registered action sample videos and candidate registered action labels as input to the simulated action generation model to obtain sample simulated action videos; use the simulated action video generation loss function to calculate the video reconstruction loss value and the constraint loss value corresponding to the sample simulated action videos; adjust the initial structural parameters based on the video reconstruction loss value and the constraint loss to obtain target structural parameters; and generate the trained simulated action generation model based on the target structural parameters.

[0072] Furthermore, the liveness detection model includes an action feature comparison model. The liveness detection device can initialize the initial structural parameters in the action feature comparison model; acquire sample authentication action videos and sample authentication action simulation videos of live sample objects; input the sample authentication action videos and sample authentication action simulation videos into the action feature comparison model to obtain the sample authentication action video fusion features of the sample authentication action videos and the sample authentication action simulation video fusion features of the sample authentication action simulation videos; use the action feature comparison loss function to calculate the multi-source feature reconstruction loss value and the liveness detection loss value of the sample authentication action video fusion features and the sample authentication action simulation video fusion features; adjust the initial structural parameters based on the multi-source feature reconstruction loss value and the liveness detection loss value to obtain the target structural parameters; and generate the trained action feature comparison model based on the target structural parameters.

[0073] It should be noted that the liveness detection device used for liveness detection and the liveness detection model training device can be the same device or different devices. Preferably, the liveness detection method and the liveness detection model training method are implemented on different liveness detection devices.

[0074] Please see also Figure 1 This diagram illustrates an example of a liveness detection method provided in this specification. When the liveness detection device detects an authentication request initiated by a target object on the identity verification page, it generates an authentication action for the user based on the target object's authentication request. Taking "looking left and right" as an example, the diagram captures a video of the target object's "looking left and right" action and sends the video to the server. The server then compares and analyzes this authentication action video with a reference action video stored by the target object during registration. The comparison yields the liveness detection result of the target object.

[0075] The liveness detection method provided in this specification will be described in detail below with reference to specific embodiments.

[0076] Please see Figure 2 This is a schematic flowchart illustrating a liveness detection method provided in the embodiments of this specification. Figure 2 As shown, the method described in the embodiments of this specification may include the following steps S102-S106.

[0077] S102, obtain the identity authentication request initiated by the target detection object for the transaction, generate authentication interaction information based on the identity authentication request, and use the authentication interaction information to instruct the target detection object to perform an authentication action;

[0078] S104, Collect the authentication action of the target detection object to generate an authentication action video;

[0079] S106, obtain the reference action video corresponding to the authentication action, compare the reference action video with the authentication action video to obtain the liveness detection result of the target detection object; the reference action video is generated based on the registration action collected when the target detection object registers the transaction.

[0080] Currently, popular liveness detection algorithms can be divided into two types. The first type is a liveness detection model based on single-frame data. This type of method takes an image as input, performs inference based on a single image, and obtains the liveness detection result. The advantage of this type of method is that it is easy to train and deploy, but the disadvantage is that its security capabilities are weak, and its defense against video display attacks is very limited. The second type is a liveness detection model based on random actions. This type of method prompts the user to complete various actions through UI interaction to judge between live and static fake objects. However, this type of method has almost no defense capability against dynamic videos generated based on static images.

[0081] To improve defenses against current generative video attacks, a liveness detection method is proposed in the embodiments of this specification. The target detection object can be a living organism with differentiated biological characteristics; this specification primarily uses humans as an example, but it can also be applied to the identification of pets such as cats and dogs. Furthermore, the target detection object is referred to as "user" in this specification, and the two terms can be used interchangeably without further explanation. This method prevents spoofing attacks by analyzing the specificity of the actions of different target detection objects (the actual video of each user performing an action differs from the algorithm-generated simulated video). See reference... Figure 3 This diagram illustrates the overall flow of a liveness detection method provided in an embodiment of this specification. Specifically, firstly, during the registration phase, a video of the user's registration action is collected. Then, based on the registration action video, multiple backup templates for other registration actions, i.e., reference action videos, are generated. Finally, during the authentication phase, an authentication action video of the target object performing the authentication action is obtained. The authentication action video and the corresponding reference action video are compared, and liveness detection and judgment are performed based on the comparison results. This fully analyzes the specificity of different users' actions, significantly improving the recall capability against video-based attack methods.

[0082] The following will provide a detailed explanation of each step:

[0083] S102, obtain the identity authentication request initiated by the target detection object for the transaction, generate authentication interaction information based on the identity authentication request, and use the authentication interaction information to instruct the target detection object to perform an authentication action;

[0084] Specifically, the authentication request initiated by the target detection object for a transaction is a request to verify whether the target detection object is a registered object. In the use of various applications and websites, users often need to log in to their accounts to obtain personalized information corresponding to the account, providing personalized services to users and preventing user information leakage. For user convenience, instead of manually entering an account and password for login and registration, the device's camera is used to take a photo, and then the facial recognition interface is called to automatically register the facial information into the platform (application software). Users can then perform facial scanning to log in. It is understood that registration can also use dual authentication of account and password and biometric authentication; the authentication request in this application mainly refers to biometric authentication.

[0085] It should be noted that the registration transaction refers to the registration of a specific application software or web program. After a user registers an account corresponding to the transaction, the user can perform identity authentication and login operations on applications and web programs within the same ecosystem of the transaction.

[0086] The authentication request can be triggered by the user's operation of the application installed on the terminal; or, the authentication request may be triggered automatically by the application installed on the terminal.

[0087] Once the application receives an authentication request, it generates authentication interaction information, which includes prompts to guide the user through the authentication process. This information can be based on the current user's account; for example, if the user has previously logged in with the account "101," the application can prompt the user to confirm whether they want to log in with "101." If the user clicks "yes," the authentication action is generated based on the "101" account information. Alternatively, the authentication interaction information can be generated in real-time using user feature information, such as a photo, and matched against the photo to generate the user's authentication action. Furthermore, the authentication interaction information includes authentication posture prompts, instructing the user to place their head or face within a preset frame, as well as text or video prompts on how to perform the authentication action.

[0088] S104, Collect the authentication action of the target detection object to generate an authentication action video;

[0089] Specifically, an authentication action video is generated by capturing video of the target object performing the authentication action using an image acquisition device. The image acquisition device can be a built-in device with image capture, storage, and transmission capabilities, or it can be an external image acquisition device. For example, when a user logs into an access control system on their mobile phone, the phone's built-in camera can be used to capture the user's authentication action, or a camera installed on the access control gate can be used. After capturing the authentication action video, simple video processing such as audio removal and image sharpness adjustment can be performed before uploading the authentication action video to a server for storage.

[0090] The authentication action can be one or more. Preferably, the authentication action is one, thereby reducing the time spent on authentication and improving authentication efficiency.

[0091] S106, obtain the reference action video corresponding to the authentication action, compare the reference action video with the authentication action video to obtain the liveness detection result of the target detection object; the reference action video is generated based on the registration action collected when the target detection object registers the transaction.

[0092] Specifically, the reference action video is generated from the registration actions captured during the registration process of the target object. This reference action video includes the registration action video of the user performing the registration actions, as well as action videos of other actions derived from the basic registration action video. Before registration, some types of actions requiring user cooperation are pre-set, such as opening the mouth, blinking, shaking the head, nodding, and looking left and right. Both authentication and registration actions are selected from these pre-set actions. Understandably, the registration action video captured during registration is the user's real video. However, if liveness detection is based solely on this single video, it is easily compromised by malicious actors. To improve the overall security of the system, multiple action videos of the same identity are generated from the registration action video, providing a solid data foundation for liveness detection.

[0093] For example, a generative adversarial network (GAN) can be used to train a video generation model. By collecting video samples from different users, key points in the videos and their corresponding key point losses are extracted. The discriminator network and the generator network are trained alternately. When the network converges, the model can generate reference action videos of other action types based on a user's registered action video. Feature comparison is performed using machine learning algorithms to determine whether the target detection object in the reference action video is consistent with the target detection object in the certified action video. This allows for the detection of liveness attacks, including attacks using mobile phone photos, paper photos, and masks, and provides a liveness detection result.

[0094] In the embodiments of this specification, by acquiring the authentication request initiated by the target detection object for a transaction, generating authentication interaction information based on the authentication request to instruct the target detection object to perform an authentication action, capturing a video of the target detection object's authentication action, and then acquiring a reference action video corresponding to the authentication action, the reference action video is compared with the authentication action video to obtain the liveness detection result of the target detection object. By comparing the differences between the authentication action video of the current target detection object's authentication action and the corresponding retained reference action video, the accuracy of liveness detection is improved by analyzing whether it is a non-liveness attack.

[0095] Please see Figure 4 This document provides a detailed flowchart of a liveness detection method for embodiments of this specification. Figure 4 As shown, the method described in the embodiments of this specification may include the following steps S202-S208.

[0096] S202, Based on the biometric information of the target detection object, determine the preferred registration action of the target detection object;

[0097] Traditional biometric registration requires users to cooperate for an extended period (e.g., rotating in various directions) to improve biometric accuracy. However, this method can lead to a decline in user experience and even users quitting midway due to the long cooperation time. To improve user registration efficiency while maintaining the performance of liveness detection and overall biometrics, a method is proposed that predicts the action most likely to reflect the user's identity information from multiple actions and then only collects data on that action. This improves data collection efficiency without affecting the performance of the biometric system.

[0098] Specifically, the biometric information of the target object is acquired, preferably an image of the user, such as a frontal photo. When the user opens the registration page, a photo of the target object is first collected as biometric information, and the preferred registration action is predicted based on this biometric information. In the specific implementation process, when predicting the preferred registration action, a preferred action prediction model can be pre-trained and used to predict the preferred registration action of the target object.

[0099] In some embodiments of this specification, the prediction of the preferred registration action based on the facial feature information of the target detection object includes:

[0100] Obtain the biometric image of the target object, extract features from the biometric image, and obtain the corresponding biometric information;

[0101] Based on the biometric information, the preferred registration action is predicted.

[0102] By extracting features from the biometric images of the target object, corresponding biometric information can be obtained, and the preferred registration action can be predicted based on this biometric information. Understandably, multiple users' biometric sample images are pre-collected, features are extracted, and the preferred registration actions corresponding to each sample are labeled. Then, the correlation between the biometric sample images and the preferred registration actions is learned to obtain a preferred action prediction model.

[0103] S204, Generate registration interaction information based on the preferred registration action, the registration interaction information being used to instruct the target detection object to perform the preferred registration action;

[0104] S206, Collect the registration action video of the target detection object;

[0105] S208, based on the registered action video, generate at least one simulated action video of a candidate registered action, and use the registered action video and the simulated action video as the reference action video.

[0106] Specifically, once the user's preferred registration action is predicted, the user only needs to perform this one registration action. A registration action video is collected based on the user's preferred registration action. Then, at least one simulated action video of the candidate registration action is generated based on the registration action video of the target detection object. A reference action video is obtained based on the registration action video and the simulated action video.

[0107] In some embodiments of this specification, generating a simulated action video of at least one alternative registered action based on the registered action video includes:

[0108] Feature extraction is performed on the registration action video to obtain the corresponding registration video features;

[0109] Based on the registered video features and at least one alternative registered action tag, a simulated action video of the alternative registered action corresponding to the number of alternative registered action tags is obtained.

[0110] By extracting features from the registered action video, registered video features are obtained. Based on these features and the candidate registered action tags to be generated, simulated action videos of the candidate registered actions corresponding to the candidate registered action tags are obtained. Specifically, a simulated video generation model can be pre-built to simulate other actions of the target detection object based on the collected registered action video.

[0111] It should be noted that when requesting users to perform authentication actions in the future, users can only be asked to perform either the alternative registration action that has been generated with a simulated action video or the registration action.

[0112] For details, see Figure 5 This diagram illustrates the registration process of a liveness detection method provided in this specification. On the registration page, the system first collects biometric information of the target object, such as a photograph. Then, it obtains the target object's preferred registration action, such as "nodding." A prompt message is generated on the registration page, prompting the user to perform the nodding action. A video of the user's registration action is then captured and stored on the server, completing the registration process. After registration, the server can generate simulated action videos of other alternative registration actions based on the registration video for subsequent authentication steps.

[0113] In some embodiments of this specification, obtaining the reference action video corresponding to the authentication action, comparing it with the authentication action video, and obtaining the liveness detection result of the target detection object includes:

[0114] Obtain the reference action video corresponding to the authentication action;

[0115] Feature extraction is performed on the authentication action video and the reference action video respectively to obtain the authentication action multi-source features corresponding to the authentication action video and the reference action multi-source features corresponding to the reference action video.

[0116] The authentication action multi-source features and the reference action multi-source features are subjected to dimensionality reduction processing to obtain the authentication action fusion features corresponding to the authentication action multi-source features and the reference action fusion features corresponding to the reference action multi-source features;

[0117] The difference between the authentication action fusion feature and the reference action fusion feature is confirmed, and the liveness detection result of the target detection object is obtained based on the difference value.

[0118] Specifically, when comparing the authentication action video and the reference action video, feature extraction is first performed, followed by dimensionality reduction. The fused features of the authentication action and the reference action are then compared to identify the differences between them. Based on these differences, a liveness detection result is obtained. Understandably, if the difference between the authentication action video and the reference action video is too large, it indicates that the target object in the authentication action video might be a fake, such as a video generated based on other content (not the registered action video of the target object). Specifically, a feature comparison model can be pre-trained; this training process is a binary classification process, and the model's final output is either "liveness authentication passed" or "liveness authentication failed."

[0119] In the embodiments of this specification, a preferred registration action for the target detection object is determined based on its biometric information. Registration interaction information is generated based on this preferred action to instruct the target detection object to perform the preferred registration action. A video of the target detection object's registration action is captured. Based on this video, a simulated action video of at least one alternative registration action is generated. The registered action video and the simulated action video are used as reference action videos to support the subsequent identity authentication process. Predicting the preferred action not only reduces the time spent on user registration but also obtains the video that best represents the user's identity characteristics, thereby improving the accuracy of liveness detection.

[0120] Please see Figure 6 This is a schematic diagram illustrating the training process of a liveness detection model provided in an embodiment of this specification. Figure 6 As shown, the method described in the embodiments of this specification may include the following steps S302-S310.

[0121] S302, Initialize the initial structural parameters in the motion prediction model;

[0122] S304, Obtain biometric sample images of live sample objects;

[0123] S306, The biometric sample image is used as input to the action prediction model to obtain the predicted identity label and the predicted preferred registration action;

[0124] S308, The action prediction loss function is used to calculate the identity tag loss value corresponding to the predicted identity tag and the action prediction loss value corresponding to the predicted preferred registration action;

[0125] S310, the initial structural parameters are adjusted based on the identity label loss value and the action prediction loss value to obtain the target structural parameters, and the trained action prediction model is generated based on the target structural parameters.

[0126] Specifically, the initial structural parameters of the action prediction model are first set, such as the model's network structure parameters and loss function. The network structure can use existing convolutional neural networks, and the loss function can be set according to actual needs; this embodiment does not impose specific limitations. Then, biometric sample images of live sample objects are acquired. These live sample objects can be people of different ages and genders. Videos of them performing registration actions are collected, and biometric sample images are extracted from the videos, while simultaneously labeling each user's identity tag. During training, the biometric sample images are input into the action prediction model to obtain predicted identity tags and predicted preferred registration actions. The identity tag loss value and action prediction loss value are calculated based on the action prediction loss function. Specifically, the loss function is: Loss... id-action=Loss id +Loss action Loss id Loss is the identity label loss value. action The loss value is used for action prediction. Based on the above model structure and loss function, the network is trained until the model converges, resulting in the trained action prediction model.

[0127] Optionally, before step S308, the method further includes:

[0128] Step a: Collect sample authentication action videos corresponding to all registration actions of the live sample object;

[0129] Step b: Based on the sample authentication action video, calculate the facial key point offset of the live sample object when performing the registration action;

[0130] Step c: Based on the facial key point offset, obtain the average key point offset corresponding to each of the registered actions;

[0131] Step d: Calculate the difference between the facial keypoint offset corresponding to each of the registered actions of the live sample object and the average keypoint offset, and take the registered action with the largest difference as the true preferred registered action of the live sample object.

[0132] In some embodiments of this specification, to obtain the user's preferred predicted registration action, the sample authentication action video corresponding to all registration actions of the live sample object is first calculated, and the average offset of key points is calculated when each action is performed. Then, the individual key point offset (facial key point offset) of each user is compared with the average key point offset, and the registration action with the largest difference between the user's individual key point offset and the average key point offset is identified. This action is taken as the user's true preferred registration action. For example, the average pupil offset value is 10 when performing the action of looking left and right, while a certain user's pupil offset value is 20. Then, the user's specific action is looking left and right.

[0133] It should be noted that the prediction of identity tags is only used in the training phase to better constrain the model, and is discarded during the deployment phase. That is, when actually using the action prediction model, it is not necessary to obtain the user's identity tags; only biometric sample images are needed.

[0134] Optionally, the action prediction model includes a feature encoding module, an identity recognition module, and an action prediction module. The step of using the biometric sample image as input to the action prediction model to obtain predicted identity labels and predicted preferred registration actions includes:

[0135] The biometric sample image is input into the feature encoding module to obtain the corresponding sample biometric information;

[0136] The biometric information of the sample is input into the identity recognition module to obtain the predicted identity label;

[0137] The sample biometric information is input into the action prediction module to obtain the predicted preferred registration action.

[0138] Specifically, the feature encoding module, also known as the feature encoder, aims to learn the compressed representation of the input. The input of the feature encoding module is a biometric sample image (a single-frame face image of the user), and the output is the corresponding sample biometric information (face features). The input of the identity recognition module is the sample biometric information (face features), and the output is the corresponding predicted identity label. The input of the action-specific prediction module is the sample biometric information (face features), and the output is the predicted preferred registration action, which is the action type that requires the user's cooperation (selecting one from {open mouth, blink, shake head, nod head, look left and right}).

[0139] Optionally, the action prediction loss function includes an identity loss function and an action loss function. The step of using the action prediction loss function to calculate the identity tag loss value corresponding to the predicted identity tag and the action prediction loss value corresponding to the predicted preferred registration action includes:

[0140] Based on the identity loss function of the identity recognition module, the deviation value between the predicted identity label and the real identity label pre-labeled by the live sample object is calculated, and the deviation value is used as the identity label loss value of the identity recognition module.

[0141] Based on the action loss function of the action prediction module, the deviation value between the predicted preferred registered action and the real preferred registered action pre-labeled by the live sample object is calculated, and the deviation value is used as the action prediction loss value of the action prediction model.

[0142] Specifically, the loss value is obtained by comparing the difference between the predicted value and the labeled true value. The deviation between the predicted identity label and the true identity label can also be represented by accuracy, such as the probability that the predicted identity label matches the true identity label. It should be noted that the prediction process is actually a classification task. The deviation between the predicted preferred registration action and the true preferred registration action can also be represented by accuracy, which indicates the correctness of the classification. For example, if the predicted preferred registration action for sample A is blinking, and the predicted preferred registration action for sample B is nodding, and the true preferred registration action for both sample A and sample B is blinking, then the accuracy is 50%, and this accuracy is used as the action prediction loss value.

[0143] Optionally, adjusting the initial structural parameters based on the identity label loss value and the action prediction loss value to obtain target structural parameters, and generating the trained action prediction model based on the target structural parameters, includes:

[0144] If the identity tag loss value does not reach the first preset value, the first initial structural parameters of the identity recognition module are adjusted based on the identity tag loss value, and the process proceeds to the step of inputting the sample biometric information into the identity recognition module to obtain the predicted identity tag, until the identity tag loss value converges to the first preset value and the first target structural parameters are obtained.

[0145] If the action prediction loss value does not reach the second preset value, the second initial structural parameters of the action prediction module are adjusted based on the action prediction loss value, and the process proceeds to the step of inputting the sample biometric information into the action prediction module to obtain the predicted preferred registration action, until the action prediction loss value converges to the second preset value and the second target structural parameters are obtained.

[0146] The trained action prediction model is generated based on the first target structure parameters and the second target structure parameters.

[0147] Understandably, the optimization process for the model aims to ensure its performance meets the desired results. Therefore, through iterative training, the initial structural parameters of the identity recognition module and the action prediction model are adjusted. When the identity label loss value converges to the first preset value and the action prediction loss value reaches the second prediction value, training ends, generating the trained action prediction model.

[0148] In the embodiments of this specification, by initializing the initial structural parameters in the action prediction model, biometric sample images of live sample objects are obtained. These biometric sample images are used as input to the action prediction model to obtain predicted identity labels and predicted preferred registration actions. The action prediction loss function is used to calculate the identity label loss value corresponding to the predicted identity label and the action prediction loss value corresponding to the predicted preferred registration action. Based on the identity label loss value and the action prediction loss value, the initial structural parameters are adjusted to obtain target structural parameters. Based on the target structural parameters, a trained action prediction model is generated. Thus, during the registration process, the action prediction model can be directly called to predict the registration action, and the registration action video can be collected for the generation of reference actions.

[0149] Please see Figure 7 This is a schematic diagram illustrating the training process of a liveness detection model provided in an embodiment of this specification. Figure 7 As shown, the method described in the embodiments of this specification may include the following steps S402-S410.

[0150] S402, Initialize the initial structural parameters in the simulated motion video generation model;

[0151] S404, Obtain the preferred registration action sample video and the sample alternative registration action tags for the live sample object;

[0152] S406, the preferred registered action sample video and the sample candidate registered action tags are used as inputs to the simulated action generation model to obtain the sample simulated action video;

[0153] S408, The video reconstruction loss value and the constraint loss value corresponding to the sample simulated action video are calculated using the simulated action video generation loss function;

[0154] S410, the initial structural parameters are adjusted based on the video reconstruction loss value and the constraint loss to obtain the target structural parameters, and the trained simulated action generation model is generated based on the target structural parameters.

[0155] Specifically, first, set the initial structural parameters of the simulated action video generation model, such as the model's network structure parameters and loss function. Then, set the candidate registered action labels for the samples, i.e., the types of actions that can be reconstructed, and collect the preferred registered action sample videos of live sample objects as training set data. The preferred registered actions for each live sample can be obtained from the trained action prediction model.

[0156] Optionally, the simulated action video generation model has a three-part structure: a video encoder, an action generator, and a constraint module. Network structure parameters are set according to the network structure of each part. The simulated action video generation loss function includes two parts: a video reconstruction loss function and a constraint loss function. The first part is the action reconstruction loss, and the second part is the multi-cue constraint loss (including optical flow loss, keypoint offset, identity feature similarity, etc.). The video reconstruction loss value is the loss when reconstructing other actions from a registered video, such as the difference between a "blink" video generated as a "look left and right" video and the real "look left and right" video. The multi-cue constraint loss is the difference in optical flow features, keypoint offsets, and identity labels between the reconstructed video and the real video.

[0157] The constraints can be adjusted according to the actual situation. The video reconstruction loss value is obtained by calculating the L2 loss function of the video frame. The L2 loss function is also known as MSE (mean square error), which is the square of the difference between the predicted value and the true value.

[0158] Therefore, the loss value of the simulated motion video generation model, Loss total =Loss recon +Loss multi-cue Loss recon The reconstruction loss, also known as the video reconstruction loss value, is called Loss. multi-cue This is the multi-threaded constraint loss, also known as the constraint loss value.

[0159] In some embodiments of this specification, the preferred sample registration action features and the sample candidate registration action tags are input into an action generator to obtain a sample simulation action video of the sample candidate registration action;

[0160] Obtain alternative registration action sample videos of the live sample object;

[0161] The sample simulated action video and the candidate registered action sample video are input into the constraint module to obtain the sample constraint parameters of the sample simulated action video and the real constraint parameters of the candidate action sample video. The constraint parameters include one or more of optical flow, key point offset and identity label.

[0162] Using the aforementioned constraint loss function, the constraint loss calculated from the sample constraint parameters and the true constraint parameters is determined. The constraint loss includes one or more of optical flow loss, keypoint offset value, and identity label similarity. Based on the video reconstruction loss function, the video reconstruction loss value of the sample simulated action video and the candidate registered action sample video is calculated.

[0163] If the video reconstruction loss value does not reach the first preset value, the first initial structural parameters of the action generator are adjusted based on the video reconstruction loss value, and the process proceeds to the step of inputting the preferred sample registration action features and the sample candidate registration action labels into the action generator to obtain the sample simulated action video of the sample candidate registration actions, until the video reconstruction loss value converges to the first preset value and the first target structural parameters are obtained.

[0164] If the constraint loss value does not reach the second preset value, the second initial structural parameters of the constraint module are adjusted based on the constraint loss value, and the process proceeds to the step of inputting the sample simulated action video and the candidate registered action sample video into the constraint module to obtain the sample constraint parameters of the sample simulated action video and the real constraint parameters of the candidate action sample video, until the constraint loss value converges to the second preset value and the second target structural parameters are obtained.

[0165] Based on the first target structure parameters and the second target structure parameters, the trained simulated action video generation model is generated.

[0166] Specifically, the video encoder takes as input the preferred registered action sample video, i.e., the user's registered video, and outputs the corresponding preferred sample registered action features (video features); the action generator takes as input the preferred sample registered action features (video features) and sample candidate registered action labels (e.g., {1: open mouth, 2: blink, 3: shake head, 4: nod head, 5: look left and right}), and outputs the sample simulated action video corresponding to the label; the constraint module takes as input the sample simulated action video and outputs multiple cue constraint losses (including optical flow, keypoints, identity, etc.). It determines whether the video reconstruction loss value converges to a first preset value; if not, iteratively trains the model until the video reconstruction loss value converges to the first preset value. Similarly, if the constraint loss value does not reach a second preset value, it adjusts the second initial structure parameters of the constraint module based on the constraint loss value and iteratively trains the model until the constraint loss value converges to the second preset value. Based on the adjusted first target structure and second target structure parameters, the trained simulated action video generation model is obtained.

[0167] In the embodiments of this specification, by initializing the initial structural parameters in the simulated action video generation model, preferred registered action sample videos and candidate registered action labels of live sample objects are obtained. These preferred registered action sample videos and candidate registered action labels are used as input to the simulated action generation model to obtain sample simulated action videos. The simulated action video generation loss function is used to calculate the video reconstruction loss value and the constraint loss value corresponding to the sample simulated action videos. Based on the video reconstruction loss value and the constraint loss, the initial structural parameters are adjusted to obtain target structural parameters. The trained simulated action generation model is then generated based on the target structural parameters. By using multi-threaded constraints, the realism of the generated videos is improved. After the simulated action generation model is trained, after a user completes registration, the registered video and the corresponding action category label can be input into the trained model to obtain backup videos of the remaining candidate registered actions.

[0168] Please see Figure 8 This is a schematic diagram illustrating the training process of a liveness detection model provided in an embodiment of this specification. Figure 8 As shown, the method described in the embodiments of this specification may include the following steps S502-S510.

[0169] S502, Initialize the initial structural parameters in the action feature comparison model;

[0170] S504, Obtain the sample authentication action video of the live sample object and the sample authentication action simulation video of the authentication action;

[0171] S506, Input the sample authentication action video and the sample authentication action simulation video into the action feature comparison model to obtain the sample authentication action video fusion feature of the sample authentication action video and the sample authentication action simulation video fusion feature of the sample authentication action simulation video.

[0172] S508, The action feature comparison loss function is used to calculate the multi-source feature reconstruction loss value and the liveness detection loss value of the sample authentication action video fusion feature and the sample authentication action simulation video fusion feature;

[0173] S510, the initial structural parameters are adjusted based on the multi-source feature reconstruction loss value and the liveness detection loss value to obtain the target structural parameters, and the trained action feature comparison model is generated based on the target structural parameters.

[0174] Specifically, the initial structural parameters of the action feature comparison model are first initialized. These parameters include the initial structure of the model. Optionally, the action feature comparison model includes a multi-source feature extractor, a feature fusion module, and an action feature comparison module. Then, sample authentication action videos of live sample objects are collected. It should be noted that the sample authentication action video can be randomly selected from all videos of registered actions performed by the live sample object. Next, a simulated video of the authentication action performed in the sample authentication action video is obtained. Features are extracted from both the real sample authentication action video and the simulated sample authentication action video of the live sample object to obtain fused features of the sample authentication action video and fused features of the sample authentication action simulation video. The multi-source feature reconstruction loss value of both is calculated, and the liveness detection result is predicted based on these features. The liveness detection loss value is obtained based on the predicted and real liveness detection results. The action feature comparison model is iteratively trained until both the multi-source feature reconstruction loss value and the liveness detection loss value meet preset conditions, resulting in the trained action feature comparison model.

[0175] In a specific implementation, the step of inputting the sample authentication action video and the sample authentication action simulation video into the action feature comparison model to obtain the sample authentication action video fusion feature of the sample authentication action video and the sample authentication action simulation video fusion feature of the sample authentication action simulation video includes:

[0176] The sample authentication action video and the sample authentication action simulation video are used as inputs to the multi-source feature extractor to obtain the sample authentication action video features of the sample authentication action video and the sample authentication action simulation video features of the sample authentication action simulation video.

[0177] The sample authentication action video features and the sample authentication action simulation video features are respectively input into the feature fusion module to obtain the sample authentication action video fusion features of the sample authentication action video features and the sample authentication action simulation video fusion features of the sample authentication action simulation video.

[0178] Specifically, the multi-source feature extractor takes sample authentication action video (user interaction video) as input and outputs sample authentication action video features, i.e., corresponding multi-source features (including optical flow, key points, identity features, etc.). The feature fusion module takes multi-source features from the sample authentication action video as input and outputs dimensionality-reduced fused features (reduced from tens of thousands of dimensions to hundreds of dimensions). It should be noted that both the actual user sample authentication action video and the simulated sample authentication action video require the above two steps for feature extraction.

[0179] Optionally, the action feature comparison loss function includes a multi-source feature reconstruction loss function and a liveness detection loss function;

[0180] The calculation of the multi-source feature reconstruction loss value and the liveness detection loss value using the action feature comparison loss function includes:

[0181] The multi-source feature reconstruction loss function is used to calculate the multi-source feature reconstruction loss value of the sample authentication action video fusion feature and the sample authentication action simulation video fusion feature;

[0182] The sample authentication action video fusion feature and the sample authentication action simulation video fusion feature are used as inputs to the action feature comparison module, and the sample liveness detection result of the liveness sample object is output. The liveness detection loss value of the sample liveness detection result is calculated based on the liveness detection loss function.

[0183] Specifically, the input to the action feature comparison module is two fused features, namely the sample-authenticated action video fused feature and the sample-authenticated action simulation video fused feature, and the output is whether the two fused features belong to the same identity (i.e., the magnitude of the difference compared with the baseline).

[0184] The loss function consists of two parts: the first part is the multi-source feature reconstruction loss from the feature fusion module, and the second part is the attack detection loss from the action feature comparison module, also known as the liveness detection loss. The purpose of calculating the multi-source feature reconstruction loss is to determine whether the multiple constraints of the generated sample authentication action simulation video are consistent with those of the real sample authentication action video. The purpose of the liveness detection loss is to ensure the model's accuracy in determining whether a person is alive.

[0185] Optionally, adjusting the initial structural parameters based on the multi-source feature reconstruction loss value and the liveness detection loss value to obtain target structural parameters, and generating the trained action feature comparison model based on the target structural parameters, includes:

[0186] If the multi-source feature reconstruction loss value does not reach the first preset value, then the first initial structural parameters of the feature fusion module are adjusted based on the multi-source feature reconstruction loss value, and the process proceeds to the step of inputting the sample authentication action video features and the sample authentication action simulation video features into the feature fusion module respectively, and reducing the dimensionality to obtain the sample authentication action video fusion features of the sample authentication action video features and the sample authentication action simulation video fusion features of the sample authentication action simulation video, until the multi-source feature reconstruction loss value converges to the first preset value, and the first target structural parameters are obtained.

[0187] If the liveness detection loss value does not reach the second preset value, the second initial structural parameters of the action feature comparison module are adjusted based on the liveness detection loss value, and the process proceeds to the step of inputting the sample simulated action video and the candidate registered action sample video into the constraint module to obtain the sample constraint parameters of the sample simulated action video and the real constraint parameters of the candidate action sample video, until the constraint loss value converges to the second preset value to obtain the second target structural parameters.

[0188] Based on the first target structure parameters and the second target structure parameters, the trained action feature comparison model is generated.

[0189] Specifically, the network is trained based on the above model structure and loss function until the model converges, that is, the loss value of multi-source feature reconstruction does not reach the first preset value and the loss value of liveness detection does not reach the second preset value, thereby obtaining the adjusted first target structure parameters and the second target structure parameters, and generating the trained action feature comparison model.

[0190] In the embodiments of this specification, by initializing the initial structural parameters in the action feature comparison model, sample authentication action videos of live sample objects and sample authentication action simulation videos of the authentication actions are obtained. These videos are then input into the action feature comparison model to obtain sample authentication action video fusion features and sample authentication action simulation video fusion features. The action feature comparison loss function is used to calculate the multi-source feature reconstruction loss value and the liveness detection loss value of the sample authentication action video fusion features and sample authentication action simulation video fusion features. Based on the multi-source feature reconstruction loss value and the liveness detection loss value, the initial structural parameters are adjusted to obtain target structural parameters. A trained action feature comparison model is then generated based on the target structural parameters. When the target detection object needs to perform identity authentication, an authentication action is performed based on randomly generated authentication interaction instructions. The collected authentication action is input into the model, and simultaneously, the server obtains a reference action video of the authentication action and inputs it into the model for feature comparison. The probability of a non-liveness attack is determined, and a liveness detection result is obtained.

[0191] The following will be combined with the appendix Figure 9-11 This specification provides a detailed description of the liveness detection device provided in the embodiments. It should be noted that the appendix... Figure 9 The liveness detection device described herein is used to perform the functions described in this manual. Figures 2-5 The methods shown in the embodiments are illustrated for ease of explanation, showing only the parts related to the embodiments of this specification. For specific technical details not disclosed, please refer to this specification. Figures 2-5 The example shown.

[0192] Please see Figure 9 This diagram illustrates a structural schematic of a liveness detection device provided in an exemplary embodiment of this specification. The liveness detection device can be implemented as all or part of a device through software, hardware, or a combination of both. The device 1 includes an authentication interaction module 11, a data acquisition module 12, and a detection module 13.

[0193] The authentication interaction module 11 is used to obtain the identity authentication request initiated by the target detection object for the transaction, generate authentication interaction information based on the identity authentication request, and instruct the target detection object to perform an authentication action.

[0194] The acquisition module 12 is used to acquire the authentication action of the target detection object to generate an authentication action video;

[0195] The detection module 13 is used to acquire a reference action video corresponding to the authentication action, and compare the reference action video with the authentication action video to obtain the liveness detection result of the target detection object; the reference action video is generated based on the registration action collected when the target detection object registers the transaction.

[0196] Optionally, the liveness detection device further includes a registration module 14, specifically used to determine the preferred registration action of the target detection object based on the biometric information of the target detection object;

[0197] Registration interaction information is generated based on the preferred registration action, and the registration interaction information is used to instruct the target detection object to perform the preferred registration action;

[0198] Collect the registration action video of the target detection object;

[0199] Based on the registered action video, generate at least one simulated action video for a candidate registered action, and use the registered action video and the simulated action video as the reference action video.

[0200] Optionally, the registration module 14 is specifically used to acquire the biometric image of the target detection object, extract features from the biometric image, and obtain the corresponding biometric information;

[0201] Based on the biometric information, the preferred registration action is predicted.

[0202] Optionally, the registration module 14 is specifically used to extract features from the registration action video to obtain corresponding registration video features;

[0203] Based on the registered video features and at least one alternative registered action tag, a simulated action video of the alternative registered action corresponding to the number of alternative registered action tags is obtained.

[0204] Optionally, the detection module 13 is specifically used to obtain a reference action video corresponding to the authentication action;

[0205] Feature extraction is performed on the authentication action video and the reference action video respectively to obtain the authentication action multi-source features corresponding to the authentication action video and the reference action multi-source features corresponding to the reference action video.

[0206] The authentication action multi-source features and the reference action multi-source features are subjected to dimensionality reduction processing to obtain the authentication action fusion features corresponding to the authentication action multi-source features and the reference action fusion features corresponding to the reference action multi-source features;

[0207] The difference between the authentication action fusion feature and the reference action fusion feature is confirmed, and the liveness detection result of the target detection object is obtained based on the difference value.

[0208] Further, refer to the appendix Figure 10 The liveness detection device shown is attached. Figure 10 The liveness detection device described herein is used to perform the functions described in this manual. Figure 6 The methods shown in the embodiments are illustrated for ease of explanation, showing only the parts related to the embodiments of this specification. For specific technical details not disclosed, please refer to this specification. Figure 6 The example shown.

[0209] Please see Figure 10 This diagram illustrates a structural schematic of a liveness detection device provided in an exemplary embodiment of this specification. The liveness detection device can be implemented as all or part of a device through software, hardware, or a combination of both. The device 2 includes a first parameter setting module 21, a first acquisition module 22, a prediction module 23, an action prediction loss calculation module 24, and a first training module 25.

[0210] The first parameter setting module 21 is used to initialize the initial structural parameters in the action prediction model;

[0211] The first acquisition module 22 is used to acquire biometric sample images of live sample objects;

[0212] The prediction module 23 is used to take the biometric sample image as input to the action prediction model to obtain the predicted identity label and the predicted preferred registration action;

[0213] Action prediction loss calculation module 24 is used to calculate the identity tag loss value corresponding to the predicted identity tag and the action prediction loss value corresponding to the predicted preferred registration action using the action prediction loss function;

[0214] The first training module 25 is used to adjust the initial structural parameters based on the identity label loss value and the action prediction loss value to obtain the target structural parameters, and to generate the trained action prediction model based on the target structural parameters.

[0215] Optionally, the prediction module 23 is specifically used to input the biometric sample image into the feature encoding module to obtain the corresponding sample biometric information;

[0216] The biometric information of the sample is input into the identity recognition module to obtain the predicted identity label;

[0217] The sample biometric information is input into the action prediction module to obtain the predicted preferred registration action.

[0218] Optionally, the action prediction loss calculation module 24 is specifically used to calculate the identity tag loss value corresponding to the predicted identity tag and the action prediction loss value corresponding to the predicted preferred registration action using an action prediction loss function, including:

[0219] Based on the identity loss function of the identity recognition module, the deviation value between the predicted identity label and the real identity label pre-labeled by the live sample object is calculated, and the deviation value is used as the identity label loss value of the identity recognition module.

[0220] Based on the action loss function of the action prediction module, the deviation value between the predicted preferred registered action and the real preferred registered action pre-labeled by the live sample object is calculated, and the deviation value is used as the action prediction loss value of the action prediction model.

[0221] Optionally, the first training module 25 is specifically used to adjust the first initial structural parameters of the identity recognition module based on the identity label loss value if the identity label loss value does not reach the first preset value, and then proceed to the step of inputting the sample biometric information into the identity recognition module to obtain the predicted identity label, until the identity label loss value converges to the first preset value and the first target structural parameters are obtained.

[0222] If the action prediction loss value does not reach the second preset value, the second initial structural parameters of the action prediction module are adjusted based on the action prediction loss value, and the process proceeds to the step of inputting the sample biometric information into the action prediction module to obtain the predicted preferred registration action, until the action prediction loss value converges to the second preset value and the second target structural parameters are obtained.

[0223] The trained action prediction model is generated based on the first target structure parameters and the second target structure parameters.

[0224] Optionally, the first acquisition module 22 is specifically used to collect sample authentication action videos corresponding to all registration actions of the live sample object;

[0225] Based on the sample authentication action video, calculate the facial key point offset of the live sample object when it performs the registration action;

[0226] Based on the facial key point offset, the average key point offset corresponding to each of the registered actions is obtained;

[0227] Calculate the difference between the facial keypoint offset corresponding to each of the registered actions of the live sample object and the average keypoint offset, and take the registered action with the largest difference as the true preferred registered action of the live sample object.

[0228] Further, refer to the appendix Figure 11 The liveness detection device shown is attached. Figure 11 The liveness detection device described herein is used to perform the functions described in this manual. Figure 7 The methods shown in the embodiments are illustrated for ease of explanation, showing only the parts related to the embodiments of this specification. For specific technical details not disclosed, please refer to this specification. Figure 7 The example shown.

[0229] Please see Figure 11 This diagram illustrates a structural schematic of a liveness detection device provided in an exemplary embodiment of this specification. The liveness detection device can be implemented as all or part of a device through software, hardware, or a combination of both. The device 3 includes a second parameter setting module 31, a second acquisition module 32, a simulation module 33, a simulated motion video generation loss calculation module 34, and a second training module 35.

[0230] The second parameter setting module 31 is used to initialize the initial structural parameters in the simulated motion video generation model;

[0231] The second acquisition module 32 is used to acquire the preferred registration action sample video and the sample alternative registration action tags of the live sample object;

[0232] The simulation module 33 is used to take the preferred registered action sample video and the sample candidate registered action tags as input to the simulated action generation model to obtain the sample simulated action video.

[0233] The simulated motion video generation loss calculation module 34 is used to calculate the video reconstruction loss value and the constraint loss value corresponding to the sample simulated motion video using the simulated motion video generation loss function.

[0234] The second training module 35 is used to adjust the initial structural parameters based on the video reconstruction loss value and the constraint loss to obtain the target structural parameters, and to generate the trained simulated action generation model based on the target structural parameters.

[0235] Optionally, the simulation module 33 is specifically used to input the preferred registered action sample video into the video encoder to obtain the preferred sample registered action features;

[0236] The preferred sample registration action features and the sample candidate registration action labels are input into the action generator to obtain the sample simulation action video of the sample candidate registration action.

[0237] Optionally, the simulated action video generation loss calculation module 34 shown is specifically used to obtain candidate registered action sample videos of the live sample object;

[0238] The sample simulated action video and the candidate registered action sample video are input into the constraint module to obtain the sample constraint parameters of the sample simulated action video and the real constraint parameters of the candidate action sample video. The constraint parameters include one or more of optical flow, key point offset and identity label.

[0239] Using the aforementioned constraint loss function, the constraint loss calculated from the sample constraint parameters and the true constraint parameters is determined. The constraint loss includes one or more of optical flow loss, keypoint offset value, and identity label similarity.

[0240] Based on the video reconstruction loss function, calculate the video reconstruction loss value of the sample simulated action video and the candidate registered action sample video.

[0241] Optionally, the second training module 35 is specifically used to adjust the first initial structural parameters of the action generator based on the video reconstruction loss value if the video reconstruction loss value does not reach the first preset value, and then proceed to the step of inputting the preferred sample registration action features and the sample candidate registration action labels into the action generator to obtain the sample simulated action video of the sample candidate registration action, until the video reconstruction loss value converges to the first preset value and the first target structural parameters are obtained;

[0242] If the constraint loss value does not reach the second preset value, the second initial structural parameters of the constraint module are adjusted based on the constraint loss value, and the process proceeds to the step of inputting the sample simulated action video and the candidate registered action sample video into the constraint module to obtain the sample constraint parameters of the sample simulated action video and the real constraint parameters of the candidate action sample video, until the constraint loss value converges to the second preset value and the second target structural parameters are obtained.

[0243] Based on the first target structure parameters and the second target structure parameters, the trained simulated action video generation model is generated.

[0244] Further, refer to the appendix Figure 12 The liveness detection device shown is attached. Figure 12 The liveness detection device described herein is used to perform the functions described in this manual. Figure 8 The methods shown in the embodiments are illustrated for ease of explanation, showing only the parts related to the embodiments of this specification. For specific technical details not disclosed, please refer to this specification. Figure 8 The example shown.

[0245] Please see Figure 12This diagram illustrates a structural schematic of a liveness detection device provided in an exemplary embodiment of this specification. The liveness detection device can be implemented as all or part of a device through software, hardware, or a combination of both. The device 4 includes a third parameter setting module 41, a third acquisition module 42, a feature fusion module 43, a multi-source feature reconstruction loss calculation module 44, and a third training module 45.

[0246] The third parameter setting module 41 is used to initialize the initial structural parameters in the action feature comparison model;

[0247] The third acquisition module 42 is used to acquire the sample authentication action video of the live sample object and the sample authentication action simulation video of the authentication action;

[0248] The feature fusion module 43 is used to input the sample authentication action video and the sample authentication action simulation video into the action feature comparison model to obtain the sample authentication action video fusion feature of the sample authentication action video and the sample authentication action simulation video fusion feature of the sample authentication action simulation video.

[0249] The multi-source feature reconstruction loss calculation module 44 is used to calculate the multi-source feature reconstruction loss value and the liveness detection loss value of the sample authentication action video fusion feature and the sample authentication action simulation video fusion feature using the action feature comparison loss function;

[0250] The third training module 45 is used to adjust the initial structural parameters based on the multi-source feature reconstruction loss value and the liveness detection loss value to obtain the target structural parameters, and to generate the trained action feature comparison model based on the target structural parameters.

[0251] Optionally, the feature fusion module 43 is specifically used to take the sample authentication action video and the sample authentication action simulation video as inputs to the multi-source feature extractor, respectively, to obtain the sample authentication action video features of the sample authentication action video and the sample authentication action simulation video features of the sample authentication action simulation video.

[0252] The sample authentication action video features and the sample authentication action simulation video features are respectively input into the feature fusion module to obtain the sample authentication action video fusion features of the sample authentication action video features and the sample authentication action simulation video fusion features of the sample authentication action simulation video.

[0253] Optionally, the multi-source feature reconstruction loss calculation module 44 is specifically used to calculate the multi-source feature reconstruction loss value of the sample authentication action video fusion feature and the sample authentication action simulation video fusion feature using a multi-source feature reconstruction loss function;

[0254] The sample authentication action video fusion feature and the sample authentication action simulation video fusion feature are used as inputs to the action feature comparison module, and the sample liveness detection result of the liveness sample object is output. The liveness detection loss value of the sample liveness detection result is calculated based on the liveness detection loss function.

[0255] Optionally, the third training module 45 is specifically used to adjust the first initial structural parameters of the feature fusion module based on the multi-source feature reconstruction loss value if the multi-source feature reconstruction loss value does not reach the first preset value, and then proceed to execute the step of inputting the sample authentication action video features and the sample authentication action simulation video features into the feature fusion module respectively, and reducing the dimensionality to obtain the sample authentication action video fusion features of the sample authentication action video features and the sample authentication action simulation video fusion features of the sample authentication action simulation video, until the multi-source feature reconstruction loss value converges to the first preset value, and the first target structural parameters are obtained;

[0256] If the liveness detection loss value does not reach the second preset value, then the second initial structural parameters of the action feature comparison module are adjusted based on the liveness detection loss value, and the process proceeds to the step of inputting the sample simulated action video and the candidate registered action sample video into the constraint module to obtain the sample constraint parameters of the sample simulated action video and the real constraint parameters of the candidate action sample video, until the constraint loss value converges to the second preset value, and the second target structural parameters are obtained.

[0257] Based on the first target structure parameters and the second target structure parameters, the trained action feature comparison model is generated.

[0258] It should be noted that the liveness detection device provided in the above embodiments is only illustrated by the division of the above functional modules when executing the liveness detection method and the liveness detection model training method. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the liveness detection device and the liveness detection method embodiments provided in the above embodiments belong to the same concept, and the implementation process is detailed in the method embodiments, which will not be repeated here.

[0259] The embodiment numbers in this specification are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0260] This specification also provides a storage medium storing a computer program, which, when executed by a processor, implements the above-described functionality. Figures 2-8 The liveness detection method described in the illustrated embodiment can be found in the following document for a detailed execution process. Figures 2-8 The specific details of the illustrated embodiments will not be elaborated here.

[0261] Please refer to Figure 13 This diagram illustrates the structure of a liveness detection device provided in an exemplary embodiment of this specification. The liveness detection device may include one or more of the following components: a processor 110, a memory 120, an input device 130, an output device 140, and a bus 150. The processor 110, memory 120, input device 130, and output device 140 may be connected via the bus 150.

[0262] Processor 110 may include one or more processing cores. Processor 110 connects to various parts of the entire liveness detection device using various interfaces and lines, and executes various functions of terminal 100 and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 120, and by calling data stored in memory 120. Optionally, processor 110 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). Processor 110 may integrate one or more of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user page, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into processor 110 and may be implemented separately using a communication chip.

[0263] The memory 120 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 120 may include non-transitory computer-readable storage medium. The memory 120 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 120 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the various method embodiments described above, etc. The operating system may be the Android system, including systems deeply developed based on the Android system, the iOS system developed by Apple Inc., including systems deeply developed based on the iOS system, or other systems.

[0264] The memory 120 can be divided into operating system space and user space. The operating system runs in the operating system space, while native and third-party applications run in user space. To ensure that different third-party applications can achieve good running performance, the operating system allocates corresponding system resources for each application. However, different application scenarios within the same third-party application have different requirements for system resources. For example, in local resource loading scenarios, third-party applications have high requirements for disk read speed; in animation rendering scenarios, third-party applications have high requirements for GPU performance. Since the operating system and third-party applications are independent of each other, the operating system often cannot promptly perceive the current application scenario of a third-party application, resulting in the operating system's inability to adapt system resources accordingly.

[0265] In order for the operating system to distinguish the specific application scenarios of third-party applications, it is necessary to establish data communication between the third-party applications and the operating system. This would allow the operating system to obtain the current scenario information of the third-party applications at any time, and then perform targeted system resource adaptation based on the current scenario.

[0266] The input device 130 is used to receive input instructions or data, and includes, but is not limited to, a keyboard, mouse, camera, microphone, or touch device. The output device 140 is used to output instructions or data, and includes, but is not limited to, a display device and a speaker. In one example, the input device 130 and the output device 140 can be combined, and the input device 130 and the output device 140 can be a touch display screen.

[0267] The touch display screen can be designed as a full-screen, curved screen, or irregularly shaped screen. It can also be designed as a combination of a full-screen and a curved screen, or a combination of an irregularly shaped screen and a curved screen; however, this specification does not limit the specific design of the embodiments.

[0268] In addition, those skilled in the art will understand that the structure of the liveness detection device shown in the above figures does not constitute a limitation on the liveness detection device. The liveness detection device may include more or fewer components than shown, or combine certain components, or have different component arrangements. For example, the liveness detection device may also include radio frequency circuits, input units, sensors, audio circuits, WiFi modules, power supplies, Bluetooth modules, etc., which will not be described in detail here.

[0269] exist Figure 13 In the liveness detection device shown, the processor 110 can be used to call the liveness detection application stored in the memory 120 and specifically perform the following operations:

[0270] Obtain the authentication request initiated by the target detection object for the transaction, generate authentication interaction information based on the authentication request, and use the authentication interaction information to instruct the target detection object to perform an authentication action.

[0271] The authentication actions of the target detection object are collected to generate an authentication action video;

[0272] Obtain the reference action video corresponding to the authentication action, and compare the reference action video with the authentication action video to obtain the liveness detection result of the target detection object; the reference action video is generated based on the registration action collected when the target detection object registers the transaction.

[0273] In one embodiment, before the processor 110 performs the operation of obtaining the reference action video corresponding to the authentication action and comparing it with the authentication action video to obtain the liveness detection result of the target detection object, it also performs the following operations:

[0274] Based on the biometric information of the target detection object, the preferred registration action of the target detection object is determined;

[0275] Registration interaction information is generated based on the preferred registration action, and the registration interaction information is used to instruct the target detection object to perform the preferred registration action;

[0276] Collect the registration action video of the target detection object;

[0277] Based on the registered action video, generate at least one simulated action video for a candidate registered action, and use the registered action video and the simulated action video as the reference action video.

[0278] In one embodiment, when the processor 110 performs the action of predicting the preferred registration based on the facial feature information of the target detection object, it specifically performs the following operations:

[0279] Obtain the biometric image of the target object, extract features from the biometric image, and obtain the corresponding biometric information;

[0280] Based on the biometric information, the preferred registration action is predicted.

[0281] In one embodiment, when the processor 110 executes the process of generating a simulated action video based on the registered action video and at least one alternative registered action, it specifically performs the following operations:

[0282] Feature extraction is performed on the registration action video to obtain the corresponding registration video features;

[0283] Based on the registered video features and at least one alternative registered action tag, a simulated action video of the alternative registered action corresponding to the number of alternative registered action tags is obtained.

[0284] The processor 110 can also be used to call the training application of the liveness detection model stored in the memory 120, and specifically perform the following operations:

[0285] Initialize the initial structural parameters in the action prediction model;

[0286] Acquire biometric sample images of live specimens;

[0287] The biometric sample image is used as input to the action prediction model to obtain the predicted identity label and the predicted preferred registration action;

[0288] The identity tag loss value corresponding to the predicted identity tag and the action prediction loss value corresponding to the predicted preferred registration action are calculated using the action prediction loss function.

[0289] The initial structural parameters are adjusted based on the identity label loss value and the action prediction loss value to obtain the target structural parameters, and the trained action prediction model is generated based on the target structural parameters.

[0290] In one embodiment, when the processor 110 executes the step of using the biometric sample image as input to the action prediction model to obtain predicted identity labels and predicted preferred registration actions, it specifically performs the following operations:

[0291] The biometric sample image is input into the feature encoding module to obtain the corresponding sample biometric information;

[0292] The biometric information of the sample is input into the identity recognition module to obtain the predicted identity label;

[0293] The sample biometric information is input into the action prediction module to obtain the predicted preferred registration action.

[0294] In one embodiment, when the processor 110 calculates the identity tag loss value corresponding to the predicted identity tag and the action prediction loss value corresponding to the predicted preferred registration action using the action prediction loss function, it specifically performs the following operations:

[0295] Based on the identity loss function of the identity recognition module, the deviation value between the predicted identity label and the real identity label pre-labeled by the live sample object is calculated, and the deviation value is used as the identity label loss value of the identity recognition module.

[0296] Based on the action loss function of the action prediction module, the deviation value between the predicted preferred registered action and the real preferred registered action pre-labeled by the live sample object is calculated, and the deviation value is used as the action prediction loss value of the action prediction model.

[0297] In one embodiment, when the processor 110 adjusts the initial structural parameters based on the identity label loss value and the action prediction loss value to obtain target structural parameters, and generates the trained action prediction model based on the target structural parameters, it specifically performs the following operations:

[0298] If the identity tag loss value does not reach the first preset value, the first initial structural parameters of the identity recognition module are adjusted based on the identity tag loss value, and the process proceeds to the step of inputting the sample biometric information into the identity recognition module to obtain the predicted identity tag, until the identity tag loss value converges to the first preset value and the first target structural parameters are obtained.

[0299] If the action prediction loss value does not reach the second preset value, the second initial structural parameters of the action prediction module are adjusted based on the action prediction loss value, and the process proceeds to the step of inputting the sample biometric information into the action prediction module to obtain the predicted preferred registration action, until the action prediction loss value converges to the second preset value and the second target structural parameters are obtained.

[0300] The trained action prediction model is generated based on the first target structure parameters and the second target structure parameters.

[0301] In one embodiment, before executing the identity loss function based on the identity recognition module, calculating the deviation between the predicted identity label and the pre-labeled real identity label of the live sample object, and using the deviation value as the identity label loss value of the identity recognition module, the processor 110 further performs the following operations:

[0302] Collect sample authentication action videos corresponding to all registration actions of the live sample object;

[0303] Based on the sample authentication action video, calculate the facial key point offset of the live sample object when it performs the registration action;

[0304] Based on the facial key point offset, the average key point offset corresponding to each of the registered actions is obtained;

[0305] Calculate the difference between the facial keypoint offset corresponding to each of the registered actions of the live sample object and the average keypoint offset, and take the registered action with the largest difference as the true preferred registered action of the live sample object.

[0306] In one embodiment, when the processor 110 invokes the training application of the liveness detection model stored in the memory 120, it specifically performs the following operations:

[0307] Initialize the initial structural parameters in the simulated motion video generation model;

[0308] Obtain the preferred registration action sample video and the sample candidate registration action tags for live sample objects;

[0309] The preferred registered action sample video and the sample candidate registered action tags are used as inputs to the simulated action generation model to obtain the sample simulated action video;

[0310] The video reconstruction loss value and the constraint loss value corresponding to the sample simulated action video are calculated using the simulated action video generation loss function.

[0311] The initial structural parameters are adjusted based on the video reconstruction loss and the constraint loss to obtain the target structural parameters, and the trained simulated action generation model is generated based on the target structural parameters.

[0312] In one embodiment, when the processor 110 executes the step of taking the preferred registered action sample video and the sample candidate registered action tags as input to the simulated action video generation model to obtain a sample simulated action video, it specifically performs the following operations:

[0313] The preferred registered action sample video is input into the video encoder to obtain the preferred sample registered action features;

[0314] The preferred sample registration action features and the sample candidate registration action labels are input into the action generator to obtain the sample simulation action video of the sample candidate registration action.

[0315] In one embodiment, when the processor 110 executes the process of generating a simulated action video based on the registered action video and at least one alternative registered action, it specifically performs the following operations:

[0316] Obtain alternative registration action sample videos of the live sample object;

[0317] The sample simulated action video and the candidate registered action sample video are input into the constraint module to obtain the sample constraint parameters of the sample simulated action video and the real constraint parameters of the candidate action sample video. The constraint parameters include one or more of optical flow, key point offset and identity label.

[0318] Using the aforementioned constraint loss function, the constraint loss calculated from the sample constraint parameters and the true constraint parameters is determined. The constraint loss includes one or more of optical flow loss, keypoint offset value, and identity label similarity.

[0319] Based on the video reconstruction loss function, calculate the video reconstruction loss value of the sample simulated action video and the candidate registered action sample video.

[0320] In one embodiment, when the processor 110 performs the calculation of the video reconstruction loss value and the constraint loss value corresponding to the sample simulated action video using the simulated action video generation loss function, it specifically performs the following operations:

[0321] Obtain alternative registration action sample videos of the live sample object;

[0322] The sample simulated action video and the candidate registered action sample video are input into the constraint module to obtain the sample constraint parameters of the sample simulated action video and the real constraint parameters of the candidate action sample video. The constraint parameters include one or more of optical flow, key point offset and identity label.

[0323] Using the aforementioned constraint loss function, the constraint loss calculated from the sample constraint parameters and the true constraint parameters is determined. The constraint loss includes one or more of optical flow loss, keypoint offset value, and identity label similarity.

[0324] Based on the video reconstruction loss function, calculate the video reconstruction loss value of the sample simulated action video and the candidate registered action sample video.

[0325] In one embodiment, when the processor 110 performs the following operations to adjust the initial structural parameters based on the video reconstruction loss value and the constraint loss to obtain target structural parameters, and generates the trained simulated action generation model based on the target structural parameters:

[0326] If the video reconstruction loss value does not reach the first preset value, the first initial structural parameters of the action generator are adjusted based on the video reconstruction loss value, and the process proceeds to the step of inputting the preferred sample registration action features and the sample candidate registration action labels into the action generator to obtain the sample simulated action video of the sample candidate registration actions, until the video reconstruction loss value converges to the first preset value and the first target structural parameters are obtained.

[0327] If the constraint loss value does not reach the second preset value, the second initial structural parameters of the constraint module are adjusted based on the constraint loss value, and the process proceeds to the step of inputting the sample simulated action video and the candidate registered action sample video into the constraint module to obtain the sample constraint parameters of the sample simulated action video and the real constraint parameters of the candidate action sample video, until the constraint loss value converges to the second preset value and the second target structural parameters are obtained.

[0328] Based on the first target structure parameters and the second target structure parameters, the trained simulated action video generation model is generated.

[0329] In one embodiment, when the processor 110 executes the process of generating a simulated action video based on the registered action video and at least one alternative registered action, it specifically performs the following operations:

[0330] Initialize the initial structural parameters in the action feature comparison model;

[0331] Obtain a video of the authentication action of a live sample object and a simulated video of the authentication action.

[0332] The sample authentication action video and the sample authentication action simulation video are input into the action feature comparison model to obtain the sample authentication action video fusion feature of the sample authentication action video and the sample authentication action simulation video fusion feature of the sample authentication action simulation video.

[0333] The multi-source feature reconstruction loss value and the liveness detection loss value of the sample authentication action video fusion feature and the sample authentication action simulation video fusion feature are calculated using the action feature comparison loss function.

[0334] The initial structural parameters are adjusted based on the multi-source feature reconstruction loss value and the liveness detection loss value to obtain the target structural parameters, and the trained action feature comparison model is generated based on the target structural parameters.

[0335] In one embodiment, when the processor 110 executes the step of inputting the sample certified action video and the sample certified action simulation video into the action feature comparison model to obtain the sample certified action video fusion feature of the sample certified action video and the sample certified action simulation video fusion feature of the sample certified action simulation video, it specifically performs the following operations:

[0336] The sample authentication action video and the sample authentication action simulation video are used as inputs to the multi-source feature extractor to obtain the sample authentication action video features of the sample authentication action video and the sample authentication action simulation video features of the sample authentication action simulation video.

[0337] The sample authentication action video features and the sample authentication action simulation video features are respectively input into the feature fusion module to obtain the sample authentication action video fusion features of the sample authentication action video features and the sample authentication action simulation video fusion features of the sample authentication action simulation video.

[0338] In one embodiment, when the processor 110 performs the calculation of the multi-source feature reconstruction loss value and the liveness detection loss value of the sample authentication action video fusion feature and the sample authentication action simulation video fusion feature using the action feature comparison loss function, the processor 110 specifically performs the following operations:

[0339] The multi-source feature reconstruction loss function is used to calculate the multi-source feature reconstruction loss value of the sample authentication action video fusion feature and the sample authentication action simulation video fusion feature;

[0340] The sample authentication action video fusion feature and the sample authentication action simulation video fusion feature are used as inputs to the action feature comparison module, and the sample liveness detection result of the liveness sample object is output. The liveness detection loss value of the sample liveness detection result is calculated based on the liveness detection loss function.

[0341] In one embodiment, when the processor 110 adjusts the initial structural parameters based on the multi-source feature reconstruction loss value and the liveness detection loss value to obtain the target structural parameters, and generates the trained action feature comparison model based on the target structural parameters, it specifically performs the following operations:

[0342] If the multi-source feature reconstruction loss value does not reach the first preset value, then the first initial structural parameter of the feature fusion module is adjusted based on the multi-source feature reconstruction loss value, and the process proceeds to the step of inputting the sample authentication action video feature and the sample authentication action simulation video feature into the feature fusion module respectively, and reducing the dimensionality to obtain the sample authentication action video fusion feature of the sample authentication action video feature and the sample authentication action simulation video fusion feature of the sample authentication action simulation video, until the multi-source feature reconstruction loss value converges to the first preset value, and the first target structural parameter is obtained.

[0343] If the liveness detection loss value does not reach the second preset value, then the second initial structural parameters of the action feature comparison module are adjusted based on the liveness detection loss value, and the process proceeds to the step of inputting the sample simulated action video and the candidate registered action sample video into the constraint module to obtain the sample constraint parameters of the sample simulated action video and the real constraint parameters of the candidate action sample video, until the constraint loss value converges to the second preset value, and the second target structural parameters are obtained.

[0344] Based on the first target structure parameters and the second target structure parameters, the trained action feature comparison model is generated.

[0345] In the embodiments of this specification, by acquiring the authentication request initiated by the target detection object for a transaction, generating authentication interaction information based on the authentication request to instruct the target detection object to perform an authentication action, capturing a video of the target detection object's authentication action, and then acquiring a reference action video corresponding to the authentication action, the reference action video is compared with the authentication action video to obtain the liveness detection result of the target detection object. By comparing the differences between the authentication action video of the current target detection object's authentication action and the corresponding retained reference action video, the accuracy of liveness detection is improved by analyzing whether it is a non-liveness attack.

[0346] Furthermore, based on the biometric information of the target object, the preferred registration action for the target object is determined. Registration interaction information is generated based on this preferred action to instruct the target object to perform the preferred registration action. A video of the target object's registration action is captured. Based on this video, a simulated action video of at least one alternative registration action is generated. The registered action video and the simulated action video are used as reference action videos to support the subsequent identity authentication process. Predicting the preferred action not only reduces the time spent on user registration but also obtains the video that best represents the user's identity characteristics, thereby improving the accuracy of liveness detection.

[0347] Furthermore, by initializing the initial structural parameters in the action prediction model, biometric sample images of live sample objects are obtained. These biometric sample images are used as input to the action prediction model to obtain predicted identity labels and predicted preferred registration actions. The action prediction loss function is used to calculate the identity label loss value corresponding to the predicted identity label and the action prediction loss value corresponding to the predicted preferred registration action. Based on the identity label loss value and the action prediction loss value, the initial structural parameters are adjusted to obtain target structural parameters. The trained action prediction model is generated based on the target structural parameters. Thus, during the registration process, the action prediction model can be directly called to predict the registration action, and the collected registration action video can be used for the generation of reference actions.

[0348] Furthermore, by initializing the initial structural parameters in the simulated action video generation model, preferred registered action sample videos and candidate registered action labels for live sample objects are obtained. These preferred registered action sample videos and candidate registered action labels are used as input to the simulated action generation model to obtain sample simulated action videos. The simulated action video generation loss function is used to calculate the video reconstruction loss value and the constraint loss value corresponding to the sample simulated action videos. Based on the video reconstruction loss value and the constraint loss, the initial structural parameters are adjusted to obtain the target structural parameters. The trained simulated action generation model is then generated based on the target structural parameters. By using multi-cue constraints, the realism of the generated videos is improved. After the simulated action generation model is trained, the registered video and the corresponding action category label can be input into the trained model after the user completes registration to obtain the backup videos of the remaining candidate registered actions.

[0349] Furthermore, by initializing the initial structural parameters in the action feature comparison model, sample authentication action videos of live sample objects and sample authentication action simulation videos of the authentication actions are obtained. These videos are then input into the action feature comparison model to obtain sample authentication action video fusion features and sample authentication action simulation video fusion features. The action feature comparison loss function is used to calculate the multi-source feature reconstruction loss value and the liveness detection loss value of the sample authentication action video fusion features and sample authentication action simulation video fusion features. Based on the multi-source feature reconstruction loss value and the liveness detection loss value, the initial structural parameters are adjusted to obtain the target structural parameters. The trained action feature comparison model is then generated based on the target structural parameters. When the target detection object needs to perform identity authentication, it performs an authentication action based on randomly generated authentication interaction instructions. The collected authentication action is input into the model, and simultaneously, the server obtains a reference action video of the authentication action and inputs it into the model for feature comparison. The probability of a non-liveness attack is determined, and a liveness detection result is obtained.

[0350] Additionally, embodiments of this specification provide a computer program product comprising a computer program that, when executed by a processor of an electronic device, enables the processor to at least perform the functions described above. Figures 1 to 8 The method provided in the illustrated embodiment.

[0351] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0352] The above-disclosed embodiments are merely preferred embodiments of this specification and should not be construed as limiting the scope of this specification. Therefore, any equivalent variations made in accordance with the claims of this specification shall still fall within the scope of this specification.

Claims

1. A method for detecting liveness, comprising: Obtain the authentication request initiated by the target detection object for the transaction, generate authentication interaction information based on the authentication request, and use the authentication interaction information to instruct the target detection object to perform an authentication action. The authentication actions of the target detection object are collected to generate an authentication action video; Obtain the reference action video corresponding to the authentication action, and compare the reference action video with the authentication action video to obtain the liveness detection result of the target detection object; the reference action video is generated based on the registration action collected when the target detection object registers the transaction; Before obtaining the reference action video corresponding to the authentication action and comparing it with the authentication action video to obtain the liveness detection result of the target detection object, the method further includes: Based on the biometric information of the target detection object, the preferred registration action of the target detection object is determined; Registration interaction information is generated based on the preferred registration action, and the registration interaction information is used to instruct the target detection object to perform the preferred registration action; Collect the registration action video of the target detection object; Based on the registered action video, generate at least one simulated action video for a candidate registered action, and use the registered action video and the simulated action video as the reference action video.

2. The method as described in claim 1, wherein determining the preferred registration action for the target detection object based on the biometric information of the target detection object includes: Obtain the biometric image of the target object, extract features from the biometric image, and obtain the corresponding biometric information; Based on the biometric information, the preferred registration action of the target detection object is predicted.

3. The method as described in claim 1, wherein generating a simulated action video of at least one candidate registered action based on the registered action video comprises: Feature extraction is performed on the registration action video to obtain the corresponding registration video features; Based on the registered video features and at least one alternative registered action tag, a simulated action video of the alternative registered action corresponding to the number of alternative registered action tags is obtained.

4. The method as described in claim 1, wherein obtaining the reference action video corresponding to the authentication action and comparing it with the authentication action video to obtain the liveness detection result of the target detection object includes: Obtain the reference action video corresponding to the authentication action; Feature extraction is performed on the authentication action video and the reference action video respectively to obtain the authentication action multi-source features corresponding to the authentication action video and the reference action multi-source features corresponding to the reference action video. The authentication action multi-source features and the reference action multi-source features are subjected to dimensionality reduction processing to obtain the authentication action fusion features corresponding to the authentication action multi-source features and the reference action fusion features corresponding to the reference action multi-source features; The difference between the authentication action fusion feature and the reference action fusion feature is confirmed, and the liveness detection result of the target detection object is obtained based on the difference value.

5. The method of claim 1, further comprising: Initialize the initial structural parameters in the action prediction model; Acquire biometric sample images of live specimens; The biometric sample image is used as input to the action prediction model to obtain the predicted identity label and the predicted preferred registration action; The identity tag loss value corresponding to the predicted identity tag and the action prediction loss value corresponding to the predicted preferred registration action are calculated using the action prediction loss function. The initial structural parameters are adjusted based on the identity label loss value and the action prediction loss value to obtain the target structural parameters. The trained action prediction model is then generated based on the target structural parameters. The action prediction model is used to determine the preferred registration action of the target detection object based on the biometric information of the target detection object.

6. The method as described in claim 5, wherein the action prediction model comprises a feature encoding module, an identity recognition module, and an action prediction module; The step of using the biometric sample image as input to the action prediction model to obtain predicted identity labels and predicted preferred registration actions includes: The biometric sample image is input into the feature encoding module to obtain the corresponding sample biometric information; The biometric information of the sample is input into the identity recognition module to obtain the predicted identity label; The sample biometric information is input into the action prediction module to obtain the predicted preferred registration action.

7. The method as described in claim 6, wherein the action prediction loss function includes an identity loss function and an action loss function; The identity tag loss value corresponding to the predicted identity tag and the action prediction loss value corresponding to the predicted preferred registration action are calculated using an action prediction loss function, including: Based on the identity loss function of the identity recognition module, the deviation value between the predicted identity label and the real identity label pre-labeled by the live sample object is calculated, and the deviation value is used as the identity label loss value of the identity recognition module. Based on the action loss function of the action prediction module, the deviation value between the predicted preferred registered action and the real preferred registered action pre-labeled by the live sample object is calculated, and the deviation value is used as the action prediction loss value of the action prediction model.

8. The method of claim 6, wherein adjusting the initial structural parameters based on the identity label loss value and the action prediction loss value to obtain target structural parameters, and generating the trained action prediction model based on the target structural parameters, comprises: If the identity tag loss value does not reach the first preset value, the first initial structural parameters of the identity recognition module are adjusted based on the identity tag loss value, and the process proceeds to the step of inputting the sample biometric information into the identity recognition module to obtain the predicted identity tag, until the identity tag loss value converges to the first preset value and the first target structural parameters are obtained. If the action prediction loss value does not reach the second preset value, the second initial structural parameters of the action prediction module are adjusted based on the action prediction loss value, and the process proceeds to the step of inputting the sample biometric information into the action prediction module to obtain the predicted preferred registration action, until the action prediction loss value converges to the second preset value and the second target structural parameters are obtained. The trained action prediction model is generated based on the first target structure parameters and the second target structure parameters.

9. The method of claim 7, further comprising, before calculating the deviation between the predicted identity label and the pre-labeled real identity label of the live sample object based on the identity loss function of the identity recognition module, and using the deviation value as the identity label loss value of the identity recognition module: Collect video recordings of all registration actions corresponding to the live sample object; Based on the sample registration action video, calculate the facial key point offset of the live sample object when it performs the registration action; Based on the facial key point offset, the average key point offset corresponding to each of the registered actions is obtained; Calculate the difference between the facial keypoint offset corresponding to each of the registered actions of the live sample object and the average keypoint offset, and take the registered action with the largest difference as the true preferred registered action of the live sample object.

10. The method of claim 1, further comprising: Initialize the initial structural parameters in the simulated motion video generation model; Obtain the preferred registration action sample video and the sample candidate registration action tags for live sample objects; The preferred registered action sample video and the sample candidate registered action tags are used as inputs to the simulated action video generation model to obtain the sample simulated action video; The video reconstruction loss value and the constraint loss value corresponding to the sample simulated action video are calculated using the simulated action video generation loss function. The initial structural parameters are adjusted based on the video reconstruction loss value and the constraint loss to obtain the target structural parameters. The trained simulated action video generation model is then generated based on the target structural parameters. The simulated action video generation model is used to generate simulated action videos of at least one candidate registered action based on the registered action video.

11. The method of claim 10, wherein the simulated motion video generation model comprises a video encoder and a motion generator; The step of using the preferred registered action sample video and the sample candidate registered action tags as input to the simulated action video generation model to obtain the sample simulated action video includes: The preferred registered action sample video is input into the video encoder to obtain the preferred sample registered action features; The preferred sample registration action features and the sample candidate registration action labels are input into the action generator to obtain the sample simulation action video of the sample candidate registration action.

12. The method of claim 11, wherein the simulated motion video generation model further includes a constraint module; The simulated motion video generation loss function includes a video reconstruction loss function and a constraint loss function; The step of calculating the video reconstruction loss value and the constraint loss value corresponding to the sample simulated action video using the simulated action video generation loss function includes: Obtain candidate registration action sample videos of the live sample object; The sample simulated action video and the candidate registered action sample video are input into the constraint module to obtain the sample constraint parameters of the sample simulated action video and the real constraint parameters of the candidate registered action sample video. The constraint parameters include one or more of optical flow, key point offset and identity label. Using the aforementioned constraint loss function, the constraint loss calculated from the sample constraint parameters and the true constraint parameters is determined. The constraint loss includes one or more of optical flow loss, keypoint offset value, and identity label similarity. Based on the video reconstruction loss function, calculate the video reconstruction loss value of the sample simulated action video and the candidate registered action sample video.

13. The method of claim 12, wherein adjusting the initial structural parameters based on the video reconstruction loss value and the constraint loss to obtain target structural parameters, and generating the trained simulated motion video generation model based on the target structural parameters, comprises: If the video reconstruction loss value does not reach the first preset value, the first initial structural parameters of the action generator are adjusted based on the video reconstruction loss value, and the process proceeds to the step of inputting the preferred sample registration action features and the sample candidate registration action labels into the action generator to obtain the sample simulated action video of the sample candidate registration actions, until the video reconstruction loss value converges to the first preset value and the first target structural parameters are obtained. If the constraint loss value does not reach the second preset value, the second initial structural parameters of the constraint module are adjusted based on the constraint loss value, and the process proceeds to the step of inputting the sample simulated action video and the candidate registered action sample video into the constraint module to obtain the sample constraint parameters of the sample simulated action video and the real constraint parameters of the candidate registered action sample video, until the constraint loss value converges to the second preset value and the second target structural parameters are obtained. Based on the first target structure parameters and the second target structure parameters, the trained simulated action video generation model is generated.

14. A method for training a liveness detection model, the liveness detection model comprising an action feature comparison model, including: Initialize the initial structural parameters in the action feature comparison model; Obtain a sample authentication action video of a live sample object and a sample authentication action simulation video of the authentication action; the sample authentication action simulation video and the sample authentication action video correspond to the same authentication action of the live sample object, and the sample authentication action simulation video is generated based on the registration action video and alternative registration action tags of the live sample object through a simulation action video generation model; The sample authentication action video and the sample authentication action simulation video are input into the action feature comparison model to obtain the sample authentication action video fusion feature of the sample authentication action video and the sample authentication action simulation video fusion feature of the sample authentication action simulation video. The multi-source feature reconstruction loss value and the liveness detection loss value of the sample authentication action video fusion feature and the sample authentication action simulation video fusion feature are calculated using the action feature comparison loss function. The initial structural parameters are adjusted based on the multi-source feature reconstruction loss value and the liveness detection loss value to obtain the target structural parameters, and the trained action feature comparison model is generated based on the target structural parameters.

15. The method as described in claim 14, wherein the action feature comparison model comprises a multi-source feature extractor, a feature fusion module, and an action feature comparison module; The step of inputting the sample authentication action video and the sample authentication action simulation video into the action feature comparison model to obtain the sample authentication action video fusion feature of the sample authentication action video and the sample authentication action simulation video fusion feature of the sample authentication action simulation video includes: The sample authentication action video and the sample authentication action simulation video are used as inputs to the multi-source feature extractor to obtain the sample authentication action video features of the sample authentication action video and the sample authentication action simulation video features of the sample authentication action simulation video. The sample authentication action video features and the sample authentication action simulation video features are respectively input into the feature fusion module to obtain the sample authentication action video fusion features of the sample authentication action video features and the sample authentication action simulation video fusion features of the sample authentication action simulation video.

16. The method of claim 15, wherein the action feature comparison loss function includes a multi-source feature reconstruction loss function and a liveness detection loss function; The calculation of the multi-source feature reconstruction loss value and the liveness detection loss value using the action feature comparison loss function to obtain the sample authentication action video fusion feature and the sample authentication action simulation video fusion feature includes: The multi-source feature reconstruction loss function is used to calculate the multi-source feature reconstruction loss value of the sample authentication action video fusion feature and the sample authentication action simulation video fusion feature; The sample authentication action video fusion feature and the sample authentication action simulation video fusion feature are used as inputs to the action feature comparison module, and the sample liveness detection result of the liveness sample object is output. The liveness detection loss value of the sample liveness detection result is calculated based on the liveness detection loss function.

17. The method of claim 16, wherein adjusting the initial structural parameters based on the multi-source feature reconstruction loss value and the liveness detection loss value to obtain target structural parameters, and generating the trained action feature comparison model based on the target structural parameters, comprises: If the multi-source feature reconstruction loss value does not reach the first preset value, then the first initial structural parameters of the feature fusion module are adjusted based on the multi-source feature reconstruction loss value, and the process proceeds to the step of inputting the sample authentication action video features and the sample authentication action simulation video features into the feature fusion module respectively, and reducing the dimensionality to obtain the sample authentication action video fusion features of the sample authentication action video features and the sample authentication action simulation video fusion features of the sample authentication action simulation video, until the multi-source feature reconstruction loss value converges to the first preset value, and the first target structural parameters are obtained. If the liveness detection loss value does not reach the second preset value, the second initial structural parameters of the action feature comparison module are adjusted based on the liveness detection loss value, and the process proceeds to the step of using the sample authentication action video and the sample authentication action simulation video as inputs to the multi-source feature extractor to obtain the sample authentication action video features of the sample authentication action video and the sample authentication action simulation video features of the sample authentication action simulation video, until the liveness detection loss value converges to the second preset value, and the second target structural parameters are obtained. Based on the first target structure parameters and the second target structure parameters, the trained action feature comparison model is generated.

18. A liveness detection device, the device comprising: The authentication interaction module is used to obtain the identity authentication request initiated by the target detection object for the transaction, generate authentication interaction information based on the identity authentication request, and instruct the target detection object to perform an authentication action. The acquisition module is used to acquire the authentication actions of the target detection object to generate an authentication action video; The detection module is used to acquire a reference action video corresponding to the authentication action, and compare the reference action video with the authentication action video to obtain the liveness detection result of the target detection object; the reference action video is generated based on the registration action collected when the target detection object registers the transaction; The registration module is used to determine the preferred registration action of the target detection object based on the biometric information of the target detection object; and to generate registration interaction information based on the preferred registration action, wherein the registration interaction information is used to instruct the target detection object to perform the preferred registration action; Collect the registration action video of the target detection object; Based on the registered action video, generate at least one simulated action video for a candidate registered action, and use the registered action video and the simulated action video as the reference action video.

19. The apparatus of claim 18, further comprising: The first parameter setting module is used to initialize the initial structural parameters in the action prediction model; The first acquisition module is used to acquire biometric sample images of live sample objects; The prediction module is used to take the biometric sample image as input to the action prediction model to obtain the predicted identity label and the predicted preferred registration action; The action prediction loss calculation module is used to calculate the identity tag loss value corresponding to the predicted identity tag and the action prediction loss value corresponding to the predicted preferred registration action using the action prediction loss function. The first training module is used to adjust the initial structural parameters based on the identity label loss value and the action prediction loss value to obtain the target structural parameters, and to generate the trained action prediction model based on the target structural parameters; the action prediction model is used to determine the preferred registration action of the target detection object based on the biometric information of the target detection object.

20. The apparatus of claim 18, further comprising: The second parameter setting module is used to initialize the initial structural parameters in the simulated motion video generation model. The second acquisition module is used to acquire the preferred registration action sample video and the sample alternative registration action tags of the live sample object; The simulation module is used to take the preferred registered action sample video and the sample candidate registered action tags as input to the simulated action video generation model to obtain the sample simulated action video. The simulated motion video generation loss calculation module is used to calculate the video reconstruction loss value and the constraint loss value corresponding to the sample simulated motion video using the simulated motion video generation loss function. The second training module is used to adjust the initial structural parameters based on the video reconstruction loss value and the constraint loss to obtain the target structural parameters, and to generate the trained simulated action video generation model based on the target structural parameters; the simulated action video generation model is used to generate simulated action videos of at least one candidate registered action based on the registered action videos.

21. A liveness detection device, the device comprising: The third parameter setting module is used to initialize the initial structural parameters in the action feature comparison model. The third acquisition module is used to acquire the sample authentication action video of the live sample object and the sample authentication action simulation video of the authentication action; the sample authentication action simulation video and the sample authentication action video correspond to the same authentication action of the live sample object, and the sample authentication action simulation video is generated based on the registration action video and alternative registration action tags of the live sample object through a simulation action video generation model; The feature fusion module is used to input the sample authentication action video and the sample authentication action simulation video into the action feature comparison model to obtain the sample authentication action video fusion feature of the sample authentication action video and the sample authentication action simulation video fusion feature of the sample authentication action simulation video. The multi-source feature reconstruction loss calculation module is used to calculate the multi-source feature reconstruction loss value and the liveness detection loss value of the sample authentication action video fusion feature and the sample authentication action simulation video fusion feature using the action feature comparison loss function. The third training module is used to adjust the initial structural parameters based on the multi-source feature reconstruction loss value and the liveness detection loss value to obtain the target structural parameters, and to generate the trained action feature comparison model based on the target structural parameters.

22. An electronic device, comprising: Processor and memory; The memory stores a computer program adapted to be loaded by the processor and to execute the steps of the method as described in any one of claims 1 to 17.

23. A storage medium storing a computer program that, when executed by a processor, implements the steps of the method as claimed in any one of claims 1 to 17.

24. A computer program product comprising: A computer program, when executed by a processor of an electronic device, causes the processor to perform the steps of the method as described in any one of claims 1 to 17.

Images