Face living body detection method and device

Abstract

The invention provides a human face living body detection method and device. The method comprises the following steps: acquiring a to-be-recognized human face image; inputting a to-be-recognized face image into the trained face anti-counterfeiting model, and obtaining a judgment result of face living body detection output by the face anti-counterfeiting model; the face anti-counterfeiting model comprises an inherent content cue word extraction module, a first text editor, an image encoder, a learnable content cue word extraction module, a second text editor, a cross-modal guide module and a classifier. According to the face in-vivo detection method and device provided by the invention, the instance-level composite prompt is generated by combining the fixed template and the learnable prompt, and feature fusion is dynamically adjusted by using the cross-modal guide module, so that the generalization ability of a face anti-counterfeiting system in an unknown field is improved, and the accuracy and reliability of face in-vivo detection are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of face recognition, and in particular to a method and device for detecting liveness of a face. Background Art

[0002] Face anti-counterfeiting technology plays a vital role in protecting facial recognition systems from various presentation attacks (such as printing attacks, replay attacks, and mask attacks), and is suitable for facial recognition, security surveillance, identity authentication, human-computer interaction, and other fields that require high security.

[0003] Although existing face anti-counterfeiting (FAS) methods have shown certain effectiveness in specific domains, when adapting to new and unseen domains, the performance will drop significantly due to the interference of domain-specific signals on facial deception cues. In order to improve the generalization ability of FAS systems in unknown domains, existing methods usually focus on learning domain-invariant representations from face images, and some methods aim to obtain a common feature space by aligning the distributions of multiple domains. However, these methods may destroy the semantic structure of the face while removing domain-specific signals, resulting in limited generalization features. It also leads to technical problems such as low performance of face liveness detection and inaccurate and unreliable detection results. Summary of the invention

[0004] The present invention provides a human face liveness detection method and device, which are used to solve the technical problems in the prior art of low human face liveness detection performance and inaccurate and unreliable detection results.

[0005] The present invention provides a method for detecting liveness of a face, comprising: Obtain a face image to be recognized; Inputting the face image to be recognized into a trained face anti-counterfeiting model, and obtaining a determination result of face liveness detection output by the face anti-counterfeiting model; The face anti-counterfeiting model includes an inherent content prompt word extraction module, a first text editor, an image encoder, a learnable content prompt word extraction module, a second text editor, a cross-modal guidance module and a classifier; The inherent content prompt word extraction module is used to extract the inherent content prompt words from the face image to be recognized; The first text editor is used to convert the inherent content prompt word into an inherent content prompt feature vector; The image encoder is used to extract image features from the face image to be identified; The learnable content extraction module is used to extract learnable content prompt words from the image features; The second text editor is used to convert the learnable content prompt words into learnable content prompt features; The cross-modal guidance module is used to fuse the image features, the inherent content prompt features and the learnable content prompt features into a comprehensive feature; The classifier is used to determine the result of face liveness detection based on the comprehensive features.

[0006] In some embodiments, the intrinsic content cue word extraction module includes a prescriptive large language model.

[0007] In some embodiments, the learnable content extraction module comprises a Q-former model.

[0008] In some embodiments, the cross-modal guidance module includes a composite language fusion unit and a visual language modality fusion unit; The composite language fusion unit is used to fuse the inherent content prompt feature and the learnable content prompt feature into a composite language fusion feature by using an attention mechanism; The visual language modality fusion unit is used to fuse the composite language fusion feature and the image feature into a comprehensive feature.

[0009] In some embodiments, the method further comprises: Get a sample image; Using the sample image, the face anti-counterfeiting model is trained by calculating a total loss function; The total loss function includes a classification loss function and a knowledge-guided loss function; The classification loss function is used to distinguish real faces from deceptive faces; the knowledge-guided loss function is used to ensure the consistency of learnable content hint features and inherent content hint features in the semantic space.

[0010] In some embodiments, the total loss function is calculated as follows: L total =W cls *L cls +W kg *L kg

[0011] Among them, L total Represents the total loss function, W cls represents the first preset weight, L cls represents the classification loss function, W kg Represents the second preset weight, L kg represents the knowledge guided loss function.

[0012] The present invention also provides a human face liveness detection device, comprising: An acquisition module, used to acquire a face image to be recognized; A detection module, used to input the face image to be identified into a trained face anti-counterfeiting model, and obtain a determination result of face liveness detection output by the face anti-counterfeiting model; The face anti-counterfeiting model includes an inherent content prompt word extraction module, a first text editor, an image encoder, a learnable content prompt word extraction module, a second text editor, a cross-modal guidance module and a classifier; The inherent content prompt word extraction module is used to extract the inherent content prompt words from the face image to be recognized; The first text editor is used to convert the inherent content prompt word into an inherent content prompt feature vector; The image encoder is used to extract image features from the face image to be identified; The learnable content extraction module is used to extract learnable content prompt words from the image features; The second text editor is used to convert the learnable content prompt words into learnable content prompt features; The cross-modal guidance module is used to fuse the image features, the inherent content prompt features and the learnable content prompt features into a comprehensive feature; The classifier is used to determine the result of face liveness detection based on the comprehensive features.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, any of the above-described methods for detecting live faces is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method for detecting liveness of a face as described above is implemented.

[0015] The present invention also provides a computer program product, including a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned methods for detecting live faces.

[0016] The face liveness detection method and device provided by the present invention generate instance-level composite prompts by combining fixed templates and learnable prompts, and dynamically adjust feature fusion using a cross-modal guidance module to improve the generalization ability of the face anti-counterfeiting system in unknown fields, thereby improving the accuracy and reliability of face liveness detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 This is one of the flow charts of the face liveness detection method provided by the present invention.

[0019] Figure 2 It is a schematic diagram of the process of preprocessing the face image provided by the present invention.

[0020] Figure 3 It is a schematic diagram of the face liveness detection system provided by the present invention.

[0021] Figure 4 It is one of the schematic diagrams of the cross-modal guidance module provided by the present invention.

[0022] Figure 5 This is the second schematic diagram of the cross-modal guidance module provided by the present invention.

[0023] Figure 6 It is a schematic diagram of the training process of the face anti-counterfeiting model provided by the present invention.

[0024] Figure 7 This is the second flow chart of the face liveness detection method provided by the present invention.

[0025] Figure 8 It is a structural schematic diagram of the face liveness detection device provided by the present invention.

[0026] Fig. 9 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0027] Recent research has begun to explore the use of large-scale visual-language models (such as CLIP) to dynamically adjust classifier weights through text features to reduce the interference of domain-specific signals and explore more generalizable visual features. However, there are two main drawbacks: first, facial categories have no semantics for the CLIP model, making it difficult to learn accurate category descriptions; second, a single form of prompt cannot depict diverse types of deception. Due to the existence of these two shortcomings, the performance of face liveness detection is low, and the detection results are inaccurate and unreliable.

[0028] Based on the above technical problems, the present invention proposes a face liveness detection method and device, which generates instance-level composite prompts by combining fixed templates and learnable prompts, and uses a cross-modal guidance module to dynamically adjust feature fusion to improve the generalization ability of the face anti-counterfeiting system in unknown fields, thereby improving the accuracy and reliability of face liveness detection.

[0029] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0030] Figure 1 FIG. 1 is a flow chart of the face liveness detection method provided by the present invention, such as Figure 1 As shown, the method includes the following: Step 101: Obtain a face image to be recognized.

[0031] Step 102: input the face image to be recognized into a trained face anti-counterfeiting model, and obtain a result of face liveness detection output by the face anti-counterfeiting model.

[0032] Specifically, the face anti-counterfeiting model includes an inherent content prompt word extraction module, a first text editor, an image encoder, a learnable content prompt word extraction module, a second text editor, a cross-modal guidance module and a classifier; the inherent content prompt word extraction module is used to extract inherent content prompt words from the face image to be identified; the first text editor is used to convert the inherent content prompt words into inherent content prompt feature vectors; the image encoder is used to extract image features (also called visual features) from the face image to be identified; the learnable content extraction module is used to extract learnable content prompt words from the image features; the second text editor is used to convert the learnable content prompt words into learnable content prompt features; the cross-modal guidance module is used to fuse the image features, the inherent content prompt features and the learnable content prompt features into comprehensive features; the classifier is used to determine the judgment result of face liveness detection based on the comprehensive features.

[0033] The inherent content prompt word extraction module, the first text editor, the image encoder, the learnable content prompt word extraction module and the second text editor constitute a composite content-aware composite prompt engineering (CCPE).

[0034] Specifically, before inputting the face image to be identified into the trained face anti-counterfeiting model, the face image to be identified may be preprocessed.

[0035] Figure 2 FIG. 1 is a schematic diagram of a process for preprocessing a face image provided by the present invention. Figure 2 As shown, the face image preprocessing may include the following steps: Step (1): Receive the input image and perform face detection. If no face is detected in the image, discard the image; otherwise, continue processing.

[0036] Step (2): All face images can be preprocessed into a uniform size (e.g., 224×224×3 pixels) to better adapt to subsequent model input requirements.

[0037] Step (3): Segment the resized face image into patches of fixed size (e.g., 14×14 pixels) to help the model learn more fine-grained facial features.

[0038] The present invention pre-processes the face image to be identified before inputting it into the trained face anti-counterfeiting model, so as to help the model learn more fine-grained facial features, thereby further improving the accuracy and reliability of face liveness detection.

[0039] The inherent content prompt word extraction module can adopt decision tree algorithm, support vector machine (SVM), predefined template and rule set, supervised learning annotation, directive large language model, etc.

[0040] Figure 3 Schematic diagram of the face liveness detection system provided by the present invention. Figure 3 As shown, in some embodiments, the intrinsic content prompt word extraction module includes a directive large language model.

[0041] The present invention, based on the rich transfer knowledge of the instruction's Large Language Model (LLM), provides specific and diverse contextual information for each face instance.

[0042] For example, by adopting the “Summarize the visual content of the image in one sentence” approach, LLM is guided to describe the image content. Each face instance will be provided with specific diverse contextual information. For each sentence, the tokenizer (·) and embedding weights of the pre-trained text encoder are used. ∈ Will Mapped to the vocabulary space. It can be expressed as:

[0043] in, represents the set of intrinsic content prompt words, Tokenizer(·) represents the token parser, Indicates i A text description of the image content, represents the embedding weight, B represents the batch size of the image, Indicates the set of intrinsic content prompt words is a three-dimensional tensor, the first dimension represents the batch size of the image. , the second dimension represents the number of intrinsic query vectors , the third dimension represents the dimension of each inherent content prompt word .

[0044] LLaVA is an advanced open source tool that acts as a generation assistant for fixed content prompts. It combines visual encoders and language models to generate image content descriptions. In addition, text descriptions based on image content The generation of happens before the training phase, thus avoiding increased computational requirements during training.

[0045] The present invention uses the rich transfer knowledge of the directive large language model to further improve the accuracy and reliability of face liveness detection.

[0046] The learnable content extraction module can adopt a Transformer model, a VGG model, a LSTM model, a Q-former model, etc.

[0047] like Figure 3 As shown, in some embodiments, the learnable content extraction module includes a Q-former model.

[0048] The present invention can learn content cues to implicitly extract the most informative visual content through Q-Former and learn the most relevant contextual representation of facial images.

[0049] For example, Q-Former is used to learn the most relevant contextual representation of face images. Given a batch of images , the image encoder first maps each image into a visual representation Then, we randomly initialize M learnable query vectors Q. We pass the visual representation and query vector through Q-former pass, which has the same architecture as BLIP-2, to obtain learnable content hint words The process can be described as:

[0050] in, represents the set of learnable content hint words for all images, represents the Q-former function, represents the query vector, represents the visual representation of the i-th image, B represents the batch size of images, Represents a set of learnable content prompt words is a three-dimensional tensor, the first dimension represents the batch size of the image , the second dimension represents the number of learnable query vectors , the third dimension represents the dimension of each learnable content prompt word .

[0051] The present invention implicitly extracts the most informative visual content through Q-Former and learns the most relevant contextual representation of facial images, thereby further improving the accuracy and reliability of face liveness detection.

[0052] Inherent content prompt words and learnable content prompt words Input text encoder and get feature vectors respectively , Intrinsic content cues are good at describing known attack types, while learnable content cues can better cover unknown attack types. In order to ensure that the specific knowledge features in the intrinsic description space are close to the unified knowledge features, the knowledge-guided loss is introduced:

[0053] in, Knowledge guided loss, B represents the batch size, and a weight factor is set ∈ The cosine distance (expressed as cos(·)) is used as a regularizer to guide the learnable content cues to approach their corresponding content-aware representations in the embedding space. The consistency between the intrinsic content cues and the learnable content cues can combine the advantages of both, better address the difficulties of diverse attacks in the FAS task, and further improve the accuracy and reliability of face liveness detection.

[0054] Figure 4 is one of the schematic diagrams of the cross-modal guidance module provided by the present invention, such as Figure 4 As shown, in some embodiments, the cross-modal guidance module includes a composite language fusion unit and a visual language modality fusion unit; The composite language fusion unit is used to fuse the inherent content prompt feature and the learnable content prompt feature into a composite language fusion feature by using an attention mechanism; The visual language modality fusion unit is used to fuse the composite language fusion feature and the image feature into a comprehensive feature.

[0055] The present invention is designed to deeply fuse features of visual and language modalities, mapping different modalities of vision and language to aligned representation spaces, including compound language fusion and visual language modal fusion. Compound language fusion achieves intra-modal complementarity of text information by designing an attention mechanism to re-weight intrinsic content cues and learnable content cues. Visual language modal fusion combines visual and language features through modal fusion to capture the complementarity of different modalities and provide more robust predictions. Since multimodal cues are often complementary, fusing visual language features as an alternative to text features as weights of visual features is usually better at capturing different aspects of a concept. In addition, the cross-modal guidance module can dynamically adjust the use of unimodal features to achieve better FAS generalization. In this way, generalized visual features are achieved by enhancing the model's understanding of content-aware text semantics, further improving the accuracy and reliability of face liveness detection.

[0056] Figure 5 is a second schematic diagram of the cross-modal guidance module provided by the present invention, such as Figure 5 As shown in the figure, an attention mechanism is designed to reweight the intrinsic content cues and the learnable content cues. First, the intrinsic content cues are represented and learnable content-cue representations Perform splicing to obtain joint text features , by applying feature average pooling , get the compressed vector Then, it is input into two fully connected layers, and the attention weights for intrinsic content cues and learnable content cues are obtained through GELU and Sigmoid-based normalization. 2. These weights are related to and Multiply and sum them to obtain the composite language fusion feature vector S. The process is expressed as:

[0057] in, represents the composite language fusion feature vector, represents the weight of the inherent content hint feature, represents the intrinsic content hint feature vector, represents the weight of the learnable content hint feature, Represents the learnable content hint feature vector.

[0058] The composite language fusion feature vector S and visual feature vector V are passed through their respective projection matrices and Map to low-dimensional space and convert into the corresponding circulant matrix Then, matrix multiplication is performed to achieve full interaction between the elements in the projection vector and the circulant matrix. This process can be expressed as:

[0059] Among them, Z represents the comprehensive feature (vector), which contains cross-modal complementary information, S represents the composite language fusion feature vector, represents the transposed vector of S, represents the projection matrix of the composite language fusion feature vector, V represents the visual feature vector, represents the transposed vector of V, The projection matrix representing the visual feature vector.

[0060] In some embodiments, the method further comprises: Get a sample image; Using the sample image, the face anti-counterfeiting model is trained by calculating a total loss function; The total loss function includes a classification loss function and a knowledge-guided loss function; The classification loss function is used to distinguish real faces from deceptive faces; the knowledge-guided loss function is used to ensure the consistency of learnable content hint features and inherent content hint features in the semantic space.

[0061] In some embodiments, the total loss function is calculated as follows: L total =W cls *L cls +W kg *L kg

[0062] Among them, L total Represents the total loss function, W cls represents the first preset weight, L cls represents the classification loss function, W kg Represents the second preset weight, L kg represents the knowledge guided loss function.

[0063] Specifically, the training process of the face anti-counterfeiting model in the present invention may include the following steps: Step (1): Preprocess all training images, adjust the image size and perform image segmentation.

[0064] Step (2): Set up the image encoder, text encoder, Q-former, cross-modal guidance module (CMG), knowledge guidance mapper Wkg , and the initial parameters of the classifier.

[0065] Step (3): Sample a batch of images from dataset D , input to the image encoder to extract image features . Also uses a prescriptive Large Language Model (LLM) to generate fixed content prompts , extracting learnable cues via Q-former . Handling intrinsic content cues via text encoders and learnable tips , extract text features and .

[0066] Step (4): Image features , Fixed content prompt feature and learnable content prompt features The input is sent to the cross-modal guidance module CMG for fusion.

[0067] Step (5): Guide the network training by calculating the loss function (for example, using the pre-trained ViT-B / 16 CLIP model as the base network), which includes the classification loss function L cls And the knowledge-guided loss function L kg The classification loss uses the cross entropy method, focusing on distinguishing real and spoofed faces; the knowledge-guided loss function uses cosine similarity to ensure the consistency of learnable content cues and inherent content cues in the semantic space. The total loss function is the weighted sum of these two parts.

[0068] Step (6): During the training process, determine whether the loss function converges. If so, terminate the training and obtain the final face anti-counterfeiting model; otherwise, proceed to the next step.

[0069] Step (7): Using the Adam optimizer, calculate the gradient of the network parameters according to the total loss L and update all model parameters.

[0070] Step (8): Return to step (3) and continue network training until the convergence condition is met or the predetermined training cycle is reached.

[0071] Figure 7 FIG. 2 is a flow chart of the method for detecting liveness of a face provided by the present invention. Figure 7 As shown in the figure, the main steps of reasoning using the face anti-counterfeiting model include: Step (1): Resize and segment the input image to complete preprocessing.

[0072] Step (2): Send the processed image to the trained image encoder to obtain the feature vector v.

[0073] Step (3): Create intrinsic content hints using a prescriptive large language model (LLM) . Through the trained Q-former, the query vector Q' is used to extract learnable content hints .

[0074] Step (4): Prompt the inherent content and tips on what to learn Input into the original text encoder respectively to obtain the corresponding text features and .

[0075] Step (5): Extract the image features v and the inherent text features and learnable text features The trained cross-modal guidance module CMG is input to achieve feature fusion and generate the fused feature representation Z.

[0076] Step (6): Use the trained classifier to process the fused feature Z and predict the result y of face liveness detection. Return the predicted result y as the final judgment of face liveness detection.

[0077] The present invention enhances the model's understanding of content-aware text semantics and explores the interference of domain-specific signals on deception clues in face anti-counterfeiting tasks, so as to achieve the learning of generalized visual features, improve the model's generalization ability in unknown fields, and further improve the accuracy and reliability of face liveness detection.

[0078] The human face liveness detection device provided by the present invention is described below. The human face liveness detection device described below and the human face liveness detection method described above can be referred to each other.

[0079] Figure 8 : is a schematic diagram of the structure of the face liveness detection device provided by the present invention, such as Figure 8 As shown, the present invention provides a human face liveness detection device, comprising: The acquisition module 801 is used to acquire the face image to be recognized; The detection module 802 is used to input the face image to be identified into the trained face anti-counterfeiting model, and obtain the judgment result of face liveness detection output by the face anti-counterfeiting model; The face anti-counterfeiting model includes an inherent content prompt word extraction module, a first text editor, an image encoder, a learnable content prompt word extraction module, a second text editor, a cross-modal guidance module and a classifier; The inherent content prompt word extraction module is used to extract the inherent content prompt words from the face image to be recognized; The first text editor is used to convert the inherent content prompt word into an inherent content prompt feature vector; The image encoder is used to extract image features from the face image to be identified; The learnable content extraction module is used to extract learnable content prompt words from the image features; The second text editor is used to convert the learnable content prompt words into learnable content prompt features; The cross-modal guidance module is used to fuse the image features, the inherent content prompt features and the learnable content prompt features into a comprehensive feature; The classifier is used to determine the result of face liveness detection based on the comprehensive features.

[0080] Specifically, the above-mentioned face liveness detection device provided in the embodiment of the present application can implement all the method steps implemented in the above-mentioned face liveness detection method embodiment, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as those of the method embodiment will not be described in detail here.

[0081] Fig. 9 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Fig. 9 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication interface 820 and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute the face liveness detection method, which includes: Obtain a face image to be recognized; Inputting the face image to be recognized into a trained face anti-counterfeiting model, and obtaining a determination result of face liveness detection output by the face anti-counterfeiting model; The face anti-counterfeiting model includes an inherent content prompt word extraction module, a first text editor, an image encoder, a learnable content prompt word extraction module, a second text editor, a cross-modal guidance module and a classifier; The inherent content prompt word extraction module is used to extract the inherent content prompt words from the face image to be recognized; The first text editor is used to convert the inherent content prompt word into an inherent content prompt feature vector; The image encoder is used to extract image features from the face image to be identified; The learnable content extraction module is used to extract learnable content prompt words from the image features; The second text editor is used to convert the learnable content prompt words into learnable content prompt features; The cross-modal guidance module is used to fuse the image features, the inherent content prompt features and the learnable content prompt features into a comprehensive feature; The classifier is used to determine the result of face liveness detection based on the comprehensive features.

[0082] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0083] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the face liveness detection method provided by the above methods, the method includes: Obtain a face image to be recognized; Inputting the face image to be recognized into a trained face anti-counterfeiting model, and obtaining a determination result of face liveness detection output by the face anti-counterfeiting model; The face anti-counterfeiting model includes an inherent content prompt word extraction module, a first text editor, an image encoder, a learnable content prompt word extraction module, a second text editor, a cross-modal guidance module and a classifier; The inherent content prompt word extraction module is used to extract the inherent content prompt words from the face image to be recognized; The first text editor is used to convert the inherent content prompt word into an inherent content prompt feature vector; The image encoder is used to extract image features from the face image to be identified; The learnable content extraction module is used to extract learnable content prompt words from the image features; The second text editor is used to convert the learnable content prompt words into learnable content prompt features; The cross-modal guidance module is used to fuse the image features, the inherent content prompt features and the learnable content prompt features into a comprehensive feature; The classifier is used to determine the result of face liveness detection based on the comprehensive features.

[0084] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the face liveness detection method provided by the above methods, the method comprising: Obtain a face image to be recognized; Inputting the face image to be recognized into a trained face anti-counterfeiting model, and obtaining a determination result of face liveness detection output by the face anti-counterfeiting model; The face anti-counterfeiting model includes an inherent content prompt word extraction module, a first text editor, an image encoder, a learnable content prompt word extraction module, a second text editor, a cross-modal guidance module and a classifier; The inherent content prompt word extraction module is used to extract the inherent content prompt words from the face image to be recognized; The first text editor is used to convert the inherent content prompt word into an inherent content prompt feature vector; The image encoder is used to extract image features from the face image to be identified; The learnable content extraction module is used to extract learnable content prompt words from the image features; The second text editor is used to convert the learnable content prompt words into learnable content prompt features; The cross-modal guidance module is used to fuse the image features, the inherent content prompt features and the learnable content prompt features into a comprehensive feature; The classifier is used to determine the result of face liveness detection based on the comprehensive features.

[0085] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0086] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0087] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting liveness of a face, characterized in that: include: Obtain a face image to be recognized; Inputting the face image to be recognized into a trained face anti-counterfeiting model, and obtaining a determination result of face liveness detection output by the face anti-counterfeiting model; The face anti-counterfeiting model includes an inherent content prompt word extraction module, a first text editor, an image encoder, a learnable content prompt word extraction module, a second text editor, a cross-modal guidance module and a classifier; The inherent content prompt word extraction module is used to extract the inherent content prompt words from the face image to be recognized; The first text editor is used to convert the inherent content prompt word into an inherent content prompt feature vector; The image encoder is used to extract image features from the face image to be identified; The learnable content extraction module is used to extract learnable content prompt words from the image features; The second text editor is used to convert the learnable content prompt words into learnable content prompt features; The cross-modal guidance module is used to fuse the image features, the inherent content prompt features and the learnable content prompt features into a comprehensive feature; The classifier is used to determine the result of face liveness detection based on the comprehensive features.

2. The method for face liveness detection according to claim 1, characterized in that: The intrinsic content prompt word extraction module includes a directive large language model.

3. The method for face liveness detection according to claim 1, characterized in that: The learnable content extraction module includes a Q-former model.

4. The method for face liveness detection according to claim 1, characterized in that: The cross-modal guidance module includes a composite language fusion unit and a visual language modality fusion unit; The composite language fusion unit is used to fuse the inherent content prompt feature and the learnable content prompt feature into a composite language fusion feature by using an attention mechanism; The visual language modality fusion unit is used to fuse the composite language fusion feature and the image feature into a comprehensive feature.

5. The method for face liveness detection according to any one of claims 1 to 4, characterized in that: The method further comprises: Get a sample image; Using the sample image, the face anti-counterfeiting model is trained by calculating a total loss function; The total loss function includes a classification loss function and a knowledge-guided loss function; The classification loss function is used to distinguish between real faces and deceptive faces; the knowledge-guided loss function is used to ensure the consistency of learnable content hint features and inherent content hint features in the semantic space.

6. The method for face liveness detection according to claim 5, characterized in that: The calculation formula of the total loss function is as follows: L total =W cls *L cls +W kg *L kg Among them, L total Represents the total loss function, W cls represents the first preset weight, L cls represents the classification loss function, W kg Represents the second preset weight, L kg represents the knowledge guided loss function.

7. A face liveness detection device, characterized in that: include: An acquisition module, used to acquire a face image to be recognized; A detection module, used to input the face image to be identified into a trained face anti-counterfeiting model, and obtain a determination result of face liveness detection output by the face anti-counterfeiting model; The face anti-counterfeiting model includes an inherent content prompt word extraction module, a first text editor, an image encoder, a learnable content prompt word extraction module, a second text editor, a cross-modal guidance module and a classifier; The inherent content prompt word extraction module is used to extract the inherent content prompt words from the face image to be recognized; The first text editor is used to convert the inherent content prompt word into an inherent content prompt feature vector; The image encoder is used to extract image features from the face image to be identified; The learnable content extraction module is used to extract learnable content prompt words from the image features; The second text editor is used to convert the learnable content prompt words into learnable content prompt features; The cross-modal guidance module is used to fuse the image features, the inherent content prompt features and the learnable content prompt features into a comprehensive feature; The classifier is used to determine the result of face liveness detection based on the comprehensive features.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method for face liveness detection as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for face liveness detection as claimed in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for face liveness detection as claimed in any one of claims 1 to 6 is implemented.

Images