Training method and apparatus for liveness detection model, and medium, electronic device and product

Abstract

The present application belongs to the technical field of artificial intelligence, and specifically relates to a training method and apparatus for a liveness detection model, and a medium, an electronic device and a product. The present application can be applied to various scenarios such as cloud technology, artificial intelligence, intelligent transportation and assisted driving. The training method comprises: occluding some regions in a sample image to obtain a mask image comprising a non-mask region, wherein the non-mask region is an unoccluded region in the mask image; acquiring encoded features of the non-mask region that are obtained by means of an encoder performing feature extraction on the non-mask region; acquiring an output image obtained by means of a decoder performing feature restoration on the encoded features of the non-mask region; and updating model parameters of the encoder and the decoder on the basis of a reconstruction loss value between the output image and the sample image. The embodiments of the present application can improve the recognition accuracy of liveness detection.

Description

Training method, device, medium, electronic equipment, and product for liveness detection model

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on August 8, 2023, with application number 2023109906078 and application name “Training method, device, medium and electronic device for liveness detection model”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application belongs to the field of artificial intelligence technology, and specifically relates to a training method for a liveness detection model, a training device for a liveness detection model, a computer-readable medium, an electronic device, and a computer program product. Background Art

[0003] Facial recognition is a biometric technology that uses facial features to identify people. It verifies user identities and provides services to verified users. However, criminals or impersonators can often interfere with facial recognition results through various cheating methods, such as photo attacks, video attacks, and model attacks, resulting in poor recognition accuracy.

[0004] Summary of the Invention

[0005] The present application provides a training method for a liveness detection model, a training device for a liveness detection model, a computer-readable medium, an electronic device, and a computer program product, the purpose of which is to improve the recognition accuracy of liveness detection.

[0006] According to one aspect of an embodiment of the present application, a training method for a liveness detection model is provided, wherein the liveness detection model includes an encoder for extracting coding features from an image, wherein the coding features are used to identify whether an object in the image is a live object; the training method includes:

[0007] Masking a portion of the sample image to obtain a mask image including a non-masked area, where the non-masked area is an unmasked area in the mask image;

[0008] Obtaining encoding features of the non-masked area obtained by the encoder performing feature extraction on the non-masked area;

[0009] Obtaining an output image obtained by restoring the coding features of the non-masked area by the decoder; the coding features of the non-masked area may also be referred to as area coding features;

[0010] The model parameters of the encoder and the decoder are updated according to the reconstruction loss value between the output image and the sample image.

[0011] According to one aspect of an embodiment of the present application, a training device for a liveness detection model is provided. The liveness detection model includes an encoder for extracting coding features from an image, wherein the coding features are used to identify whether an object in the image is a live object. The training device includes:

[0012] a masking module configured to mask a portion of the sample image to obtain a mask image including a non-masked area, where the non-masked area is an unmasked area in the mask image;

[0013] an encoding module configured to obtain encoding features of the non-masked area obtained by the encoder performing feature extraction on the non-masked area;

[0014] a decoding module configured to obtain an output image obtained by restoring the encoded features of the non-masked area by the decoder;

[0015] An updating module is configured to update model parameters of the encoder and the decoder according to a reconstruction loss value between the output image and the sample image.

[0016] In some embodiments of the present application, based on the above technical solution, the sample images include living sample images corresponding to living objects and non-living sample images corresponding to non-living objects; and the training device for the living detection model further includes:

[0017] a sample pair acquisition module configured to combine the two sample images into a sample pair, wherein the sample pair includes a positive sample pair consisting of two living sample images or a negative sample pair consisting of one living sample image and one non-living sample image;

[0018] a feature comparison module, configured to compare the encoding features of the two sample images in the sample pair to obtain the feature similarity of the sample pair;

[0019] an error determination module configured to determine a contrast loss value of the sample image based on the feature similarity, the contrast loss value being used to represent an ability of the encoder to extract similar encoding features for a plurality of the sample images corresponding to a living object;

[0020] A parameter updating module is configured to update the model parameters of the encoder according to the contrast loss value.

[0021] In some embodiments of the present application, based on the above technical solution, the parameter update module is further configured to: obtain an error threshold used to represent the degree of convergence of the encoder; when the reconstruction loss value is less than the error threshold, update the model parameters of the encoder according to the contrast loss value.

[0022] In some embodiments of the present application, based on the above technical solution, the error determination module is further configured to: assign different weight coefficients to the positive sample pairs and the negative sample pairs, respectively; and determine the contrast loss value of the sample image based on the weight coefficient and the feature similarity.

[0023] In some embodiments of the present application, based on the above technical solution, the weight coefficient of the positive sample pair is greater than the weight coefficient of the negative sample pair.

[0024] In some embodiments of the present application, based on the above technical solution, the positive sample pairs include same-domain positive sample pairs and non-same-domain positive sample pairs, the same-domain positive sample pairs are composed of two live image samples with the same domain label, and the non-same-domain positive sample pairs are composed of two live image samples with different domain labels, and the domain label is used to indicate the category information of the live image sample; the weight coefficient of the non-same-domain positive sample pair is greater than the weight coefficient of the same-domain positive sample pair.

[0025] In some embodiments of the present application, based on the above technical solution, the shielding module includes:

[0026] An image segmentation module is configured to segment a sample image into a plurality of tiles of the same size;

[0027] The block masking module is configured to randomly mask a number of the blocks in the sample image to obtain a mask image, wherein the mask image includes a mask area consisting of the masked blocks and a non-mask area consisting of the unmasked blocks.

[0028] In some embodiments of the present application, based on the above technical solution, the number of tiles in the masked area is greater than the number of tiles in the non-masked area.

[0029] In some embodiments of the present application, based on the above technical solution, the encoding module includes:

[0030] a first mapping module configured to perform linear mapping on each image block in the non-masked area to obtain a semantic feature vector of the image block;

[0031] A first order determination module is configured to determine the encoding order of the image blocks according to the distribution positions of the image blocks in the sample image;

[0032] A first feature extraction module is configured to obtain encoding features of the image blocks obtained by the encoder performing feature extraction on the semantic feature vectors of the respective image blocks;

[0033] The feature splicing module is configured to splice the coding features of each of the image blocks into the coding features of the non-masked area according to the coding order of the image blocks.

[0034] In some embodiments of the present application, based on the above technical solution, the encoding module includes:

[0035] a second mapping module, configured to perform linear mapping on each image block in the non-mask area to obtain a semantic feature vector of the non-mask area;

[0036] A first position determination module is configured to determine a position feature vector of the non-mask area according to a distribution position of the image block in the sample image;

[0037] The second feature extraction module is configured to obtain the encoding features of the non-masked area obtained by the encoder performing feature extraction on the semantic feature vector of the non-masked area and the position feature vector of the non-masked area.

[0038] In some embodiments of the present application, based on the above technical solution, the decoding module includes:

[0039] a second order determination module, configured to determine the encoding order of the image blocks according to the distribution positions of the image blocks in the sample image;

[0040] a feature input module configured to concatenate the coding features of the non-masked area and the coding features of the masked area according to the coding order and input the concatenated features into a decoder;

[0041] The first feature restoration module is configured to obtain an output image obtained by the decoder performing feature restoration on the coding features of the non-masked area and the coding features of the masked area.

[0042] In some embodiments of the present application, based on the above technical solution, the decoding module includes:

[0043] a second position determination module, configured to determine a position feature vector of the non-mask area according to a distribution position of the image block in the sample image;

[0044] The second feature restoration module is configured to obtain an output image obtained by the decoder performing feature restoration on the encoding features of the non-masked area and the position feature vector of the non-masked area.

[0045] In some embodiments of the present application, based on the above technical solution, the liveness detection model further includes a classifier for identifying liveness images based on the coding features; and the training device for the liveness detection model further includes:

[0046] a third feature extraction module, configured to obtain encoding features of the sample image obtained by the encoder performing feature extraction on the unobstructed sample image;

[0047] a feature classification module configured to obtain a recognition result obtained by the classifier through classification processing of the coded features of the sample image, wherein the recognition result is used to indicate whether the sample image is a living body image;

[0048] A parameter updating module is configured to update the model parameters of the encoder and the classifier according to the recognition result.

[0049] According to one aspect of an embodiment of the present application, a computer-readable medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the training method of the living body detection model in the above technical solution is implemented.

[0050] According to one aspect of an embodiment of the present application, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the executable instructions to implement a training method for a liveness detection model as in the above technical solution.

[0051] According to one aspect of an embodiment of the present application, a computer program product is provided, including a computer program, which, when executed by a processor, implements the training method of the living body detection model in the above technical solution.

[0052] In the technical solution provided in the embodiment of the present application, a mask image including a non-masked area is obtained by masking a portion of a sample image, encoding features of the non-masked area are obtained by extracting features from the non-masked area by the encoder, and an output image is obtained by restoring the encoding features of the non-masked area by the decoder. The model parameters of the encoder and decoder are updated based on the reconstruction loss value between the output image and the sample image. The embodiment of the present application uses mask reconstruction to train the encoder's feature extraction capabilities, which can improve the accuracy of the liveness detection model in identifying whether a sample image corresponds to a live object.

[0053] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] FIG1 shows a block diagram of a system architecture in which the technical solution of the present application is applied.

[0055] FIG2 shows a training method for a liveness detection model in one embodiment of the present application.

[0056] FIG3 shows a schematic diagram of the training process of a liveness detection model in one embodiment of the present application.

[0057] FIG4 shows a method for training a liveness detection model based on contrastive learning in one embodiment of the present application.

[0058] FIG5 shows a method for training a liveness detection model based on encoding sequence in one embodiment of the present application.

[0059] FIG6 shows a method for training a liveness detection model based on block position features in an application scenario of the present application.

[0060] FIG7 shows a schematic diagram showing the principle of training a liveness detection model in an application scenario according to an embodiment of the present application.

[0061] FIG8 schematically shows a structural block diagram of a training device for a living body detection model provided in an embodiment of the present application.

[0062] FIG9 schematically shows a block diagram of a computer system structure of an electronic device suitable for implementing an embodiment of the present application. DETAILED DESCRIPTION

[0063] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art.

[0064] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner.In the following description, many specific details are provided so as to provide a full understanding of the embodiments of the present application. However, it will be appreciated by those skilled in the art that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps etc. can be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0065] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0066] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, while others may be combined or partially combined. Therefore, the actual execution order may vary depending on the actual situation.

[0067] In the specific implementation of this application, relevant data such as user facial images are involved. When the various embodiments of this application are applied to specific products or technologies, user permission or consent must be obtained, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0068] In the related technologies of this application, the liveness detection method based on deep learning may include a liveness detection method based on traditional binary classification and a liveness detection algorithm with added meta-learning.

[0069] Deep learning liveness detection methods based on traditional binary classification typically rely solely on binary classification labels to supervise the deep network, requiring complex design of the network structure (i.e., feature extractor) to improve network recognition accuracy. While these methods offer simple supervision and intuitively align with task logic, they typically generalize poorly to unknown attack samples and data captured in scenes or on devices that differ from the training data. These methods typically require a large and diverse set of data samples to train the algorithm model, resulting in high data collection and annotation costs. Furthermore, the design of these complex models relies heavily on intuition and lacks interpretability.

[0070] Liveness detection algorithms that incorporate meta-learning are based on binary classification supervision and feature a unique network structure designed to enhance feature learning and generalization. While these methods generally outperform previous liveness detection methods in generalization, their network design is often complex, making algorithm training and convergence challenging and requiring sophisticated training methods to achieve good results. For businesses, these algorithms are difficult to iterate and lack stability, posing certain risks to algorithm iteration.

[0071] In the face liveness detection business scenario, the amount of data available for optimizing deep learning models is limited, and malicious users are constantly challenging the liveness detection systems used in the business with new liveness attack methods. In this situation, the continuous incremental attacks pose a severe challenge to the generalization and feature extraction capabilities of the model algorithm.

[0072] In response to the problems existing in the above-mentioned related technologies, the embodiment of the present application first designs a pre-training framework for image mask reconstruction based on the idea of ​​self-supervised learning. On the basis of the conventional binary classification supervised liveness detection deep learning framework, the model's ability to extract face liveness-related features is improved through self-supervised pre-training, thereby improving the recognition accuracy of face liveness detection.

[0073] FIG1 shows a block diagram of a system architecture in which the technical solution of the present application is applied.

[0074] As shown in Figure 1, the system architecture applying the technical solution of the present application may include a terminal device 110 and a server 130. The terminal device 110 may include various electronic devices such as smart phones, tablet computers, laptops, desktop computers, smart speakers, smart wearable devices, smart car devices, smart payment terminals, etc. The server 130 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. Between the terminal device 110 and the server 130, various connection types of communication media for providing communication links may be included, such as a wired communication link or a wireless communication link.

[0075] The liveness detection model 120 is a model used to detect whether a business object is a real user. For example, when performing face recognition on a business object, liveness detection is performed on a captured image to determine whether the image is a face image of a real user.

[0076] In one application scenario of an embodiment of the present application, the liveness detection model 120 can be pre-deployed on the server 130, and the server 130 can train the liveness detection model 120. During the model training process, the loss error can be determined based on the recognition results of the liveness detection model 120 for the training samples, and then the model parameters of the liveness detection model 120 can be iteratively updated based on the loss error. Through continuous training, the loss error of the model can be gradually reduced, thereby improving the recognition accuracy of the model.

[0077] After the liveness detection model 120 is trained, it can provide liveness detection services to the terminal device 110. For example, the terminal device 110 can upload an image to be recognized to the server 130. The liveness detection model 120 deployed on the server 130 will process the image to be recognized and output a recognition result. The server 130 will then return the recognition result to the terminal device 110 to determine whether the image to be recognized is a real user's face image.

[0078] In other application scenarios, the trained liveness detection model 120 can also be deployed directly on the terminal device 110, allowing the terminal device 110 to run the liveness detection model locally. When liveness detection is required, the terminal device 110 can input the image to be recognized into the trained liveness detection model 120. The liveness detection model 120 will perform recognition processing on the image to be recognized and output a recognition result to determine whether the image to be recognized is a real user's face image.

[0079] The liveness detection model provided in the embodiments of this application can be applied to a variety of different online business scenarios, including cloud technology, artificial intelligence, smart transportation, assisted driving, and other scenarios. For example, the facial authentication function involved in social software or instant messaging software is mainly used for user real-name authentication, account unblocking appeals, and other operations; the driver remote authentication process involved in online car-hailing software is mainly used to determine whether the current driver is a real person; the facial recognition access control system in smart access control systems and the identity verification involved in account unblocking in gaming services, etc.

[0080] The following is a detailed description of the technical solutions provided in this application, including the training method of the liveness detection model, the training device of the liveness detection model, the computer-readable medium, the electronic device, and the computer program product, in combination with specific implementation methods.

[0081] Figure 2 shows a training method for a liveness detection model in an embodiment of the present application. The training method for the liveness detection model can be executed separately by the terminal device or server shown in Figure 1, or can be executed jointly by the terminal device and the server. The embodiment of the present application uses the training method executed by the server as an example to illustrate. As shown in Figure 2, the liveness detection model includes an encoder for extracting coding features from an image, and also includes a decoder for processing the output of the encoder. The training method for the liveness detection model can include the following steps S210 to S240.

[0082] S210: masking a portion of the sample image to obtain a mask image including a non-masked area. The non-masked area is an unmasked area in the mask image, and the masked area can be recorded as a masked area.

[0083] S220: Obtain the regional coding features obtained by extracting features from the non-mask area by the encoder. The regional coding features may also be referred to as coding features of the non-mask area. In one embodiment of the present application, when S220 is executed during the training process of the encoder and decoder, the mask image may be first input into the encoder, and then the output of the encoder may be obtained to obtain the coding features of the non-mask area; and in another embodiment of the present application, only the non-mask areas in the mask image may be input into the encoder respectively, and then the output of the encoder may be obtained to obtain the coding features of the non-mask area. For example, the image blocks corresponding to the non-mask areas may be input into the encoder to obtain the coding features of the non-mask areas corresponding to these image blocks.

[0084] S230: Obtain an output image obtained by restoring the regional coding features by the decoder. The feature restoration process is the process in which the decoder processes the coding features of the non-masked area to obtain an output image. The feature restoration process will also be continuously optimized as the encoder is trained and optimized. The decoder that has been trained will be able to process the coding features of the non-masked area output from the trained and optimized encoder to obtain an image corresponding to the mask image that can more accurately detect living objects. The output image obtained after feature restoration can also be called a reconstructed image, which can be understood as reconstructing the features of the occluded part of the sample image to form a reconstructed image that can be compared with the original sample image.

[0085] S240: Update the model parameters of the encoder and decoder according to the reconstruction loss value between the output image and the sample image.

[0086] In the training method of the liveness detection model provided in the embodiment of the present application, the feature extraction capability of the encoder is trained by using mask reconstruction, which can improve the accuracy of the liveness detection model in identifying whether the sample image corresponds to a live object. After completing the update of the model parameters of the encoder and decoder, any input image can be processed by the updated encoder and decoder, and the encoded features of the non-masked area of ​​the input image are obtained by the optimized encoder. The decoder then performs feature restoration on the encoded features of the input image to obtain an output image of the input image. Based on the output image, more accurate liveness detection can be achieved for the input image. In some cases, the reconstruction loss value can also be referred to as the reconstruction loss error.

[0087] Figure 3 shows a schematic diagram of the training process of a liveness detection model in one embodiment of the present application. The liveness detection model in this embodiment of the present application may include an encoder 301 and a classifier 302. Encoder 301 is used to extract encoding features from an input image, and classifier 302 is used to identify whether the input image is a liveness image based on the encoding features. Liveness images refer to images captured of living subjects, such as images of real users' faces.

[0088] This embodiment of the present application is primarily used to pre-train encoder 301 in a liveness detection model. Pre-training refers to the process of obtaining optimal neural network initialization parameters through specialized training methods before formal neural network training. By pre-training encoder 301, this embodiment of the present application can improve encoder 301's feature extraction capabilities, thereby increasing the liveness detection model's liveness image recognition accuracy.

[0089] As shown in Figure 3, before pre-training the encoder 301, part of the sample image 303 can be masked to obtain a mask image 304 including a non-mask area. The non-mask area is the unmasked area in the mask image 304, and the masked area in the mask image 304 is recorded as the mask area. In an embodiment of the present application, a plurality of mask tiles can be used to randomly mask the sample image to form a mask image with randomly distributed mask areas. The mask tile can be a tile with specified pixel content, for example, a pure white or pure black tile. The mask tiles that form occlusion on different areas of a mask image can have the same size or different sizes.

[0090] The mask image 304 is input to the encoder 301, which extracts features from the non-masked areas of the mask image to obtain encoded features 305 of the non-masked areas. Encoded features 305 are deep features used to characterize the image content and can be represented as feature vectors with a specified dimension.

[0091] The encoded features 305 are then input to a decoder 306 corresponding to the encoder 301. The decoder 306 then performs feature restoration on the encoded features 305 of the non-masked region to produce an output image 307. Based on the encoded features 305 of the non-masked region, the decoder 306 can restore the features of the non-masked region to obtain the reconstructed image content of the non-masked region. Simultaneously, the decoder 306 can also perform feature restoration on the masked region based on the encoded features 305 of the non-masked region to obtain the reconstructed image content of the masked region. The reconstructed image contents of the non-masked region and the masked region together constitute the output image 307.

[0092] Finally, the model parameters of the encoder 301 and the decoder 306 are updated according to the reconstruction loss value between the output image 307 and the sample image 303 (ie, the reconstruction loss error 308 ).

[0093] By iteratively executing the above pre-training process, the model parameters of the encoder 301 and the decoder 306 can be continuously updated, the ability of the encoder 301 to extract deep features from the mask image is continuously improved, and the ability of the decoder 306 to restore the output image from the encoded features is continuously improved.

[0094] In one embodiment of the present application, encoder 301 may employ the ViT model, also known as the Vision Transformer. The ViT model applies the self-attention-based Transformer model from the NLP field to image tasks. Compared to traditional convolutional neural network models for image tasks, the ViT model is more effective and more cost-effective on large datasets.

[0095] The ViT model as a whole can include multiple parts such as the image block embedding unit Patch Embedding, the positional encoding unit Positional Encoding, and the transform encoder Transformer Encoder.

[0096] The patch embedding unit is used to convert the input two-dimensional image into a one-dimensional vector suitable for encoding. The image is first converted into a sequence of multiple image patches. A linear transformation is then used to convert the patch sequence into a one-dimensional vector, similar to word embeddings in natural language processing (NLP). A classification vector is appended to the resulting vector after patch embedding to learn category information when training the Transformer Encoder. Assuming the image is divided into N patches, the input to the Transformer Encoder will result in N vectors. A learnable embedding vector is manually added as the category vector for classification. This is then fed into the Transformer Encoder along with the other patch embedding vectors. Finally, the first learnable embedding vector appended is used as the category prediction result.

[0097] Positional encoding is used to preserve the spatial position information of an image. Unlike CNNs, the Transformer loses the position information of the original image due to image block embedding. Therefore, positional encoding is needed to provide additional position information.

[0098] The Transformer Encoder consists of alternating multi-head self-attention layers (MSA) and multi-layer perceptron blocks (MLP). Layer normalization is applied before each block and residual connection is applied after each block.

[0099] In one embodiment of the present application, the decoder 306 may select a ViT model with a smaller parameter scale than the encoder 301. The parameter scale may include Patch Size, Layers, Hidden Size, MLP size, and Heads. Patch Size is the tile size of the input model, for example, it may include 14x14, 16x16, 32x32, etc. Layers is the number of times the Encoder Block is repeatedly stacked in the Transformer Encoder, for example, it may include 12, 24, 32. Hidden Size is the vector length corresponding to each vector after passing through the Embedding layer, for example, it may include 768, 1024, 1280. MLP size is the number of nodes in the first fully connected MLP Block (Multi-layer Perceptron module) in the Transformer Encoder, which is four times the Hidden Size. Heads represents the number of heads in the Multi-Head Attention in the Transformer Encoder, for example, it may include 12 or 16.

[0100] Decoder 306 uses a more lightweight model than encoder 301, which can improve the decoder's convergence speed, prevent the decoder training process from occupying too many computing resources, and save time and computing resources. For example, the total number of model parameters for encoder 301 is 307M, while the total number of model parameters for decoder 306 is 86M. While ensuring the high training accuracy of encoder 301, decoder 306 can complete training more quickly, improving the overall training efficiency of the model.

[0101] In one embodiment of the present application, the model parameters may be iteratively updated based on a gradient descent algorithm and a back propagation algorithm.

[0102] Gradient descent is an optimization algorithm used to find parameter values ​​that minimize a loss function. The basic idea behind gradient descent is to find the minimum value of the loss function by moving the parameters in the opposite direction of the gradient based on the current parameter's gradient. The principle behind gradient descent can be summarized as follows: starting from a point in a high-dimensional space, the algorithm moves step by step toward the optimal solution, following the derivative of the loss function and moving in the direction of fastest decrease, ultimately arriving at the optimal solution.

[0103] Backpropagation is a formalized gradient descent algorithm used to train neural networks. The basic idea behind backpropagation is to propagate the gradient of the output layer back to the hidden layers to calculate the gradient of each layer and then apply the gradient to the model parameters in the hope of minimizing the loss function. Backpropagation combines the gradient descent algorithm with the solution of negative gradients. The principle behind backpropagation is to propagate the error from the output layer of the neural network to the input layer, calculating the partial derivative of the error with respect to each parameter at each layer, and updating the weight parameters using gradient descent to minimize the error, thereby improving the accuracy of the model.

[0104] In one embodiment of the present application, the encoder's main algorithmic logic uses a ViT model to segment a facial image into non-overlapping image tokens, then randomly occludes them at a rate of approximately 75%, leaving only a small number of image tokens. These remaining image tokens are then used as the encoder's input, and the final output is an encoded representation of these visible tiles, dimensionally consistent with the input. During the pre-training phase, its primary task is to extract and encode facial features as input to the subsequent decoder. In subsequent stages, the encoder module will serve as a feature extractor for liveness detection tasks and connect to a classifier for classification.

[0105] The decoder's primary purpose is to decode the previously encoded representation of visible patches, recovering the relevant features of the original image and calculating a loss function to optimize the model's parameters. Its input is the encoder's output and a learnable token. Previously occluded tokens are replaced with learnable tokens and fed into the decoder in their original order. The decoder then recovers the pixel features of the original image, with the target values ​​derived directly from the original image.

[0106] In one embodiment of the present application, the reconstruction loss value involved is calculated by a loss function, which can be a reconstruction loss function. The reconstruction loss function L used to calculate the reconstruction loss value is rec as follows.

[0107] Among them, T v represents the visible image token, that is, the non-masked area of ​​the mask image; and T m Represents the masked image token, that is, the mask area of ​​the mask image; T i Represents the original sample image; D represents the decoder, G represents the encoder; the indicative function It is used to indicate that the reconstruction loss is only calculated on the masked image tokens, n represents the total number of image tokens, and θ represents the encoder / decoder parameters.

[0108] In one embodiment of the present application, after updating the model parameters of the encoder and decoder according to the reconstruction loss value between the output image and the sample image, the encoder 301 and the classifier 302 can be trained together, thereby further improving the feature extraction capability of the encoder 301 while improving the classification and recognition capability of the classifier 302.

[0109] The method for jointly training the encoder 301 and the classifier 302 may include: obtaining the encoding features of the sample image obtained by the encoder performing feature extraction on the unobstructed sample image; obtaining the recognition result obtained by the classifier performing classification processing on the encoding features of the sample image, the recognition result being used to indicate whether the sample image is a living image; and updating the model parameters of the encoder and the classifier according to the recognition result.

[0110] After completing encoder pre-training, the fine-tuning phase begins. During this phase, the downstream task image is directly fed into the encoder without occlusion to obtain encoded features. Global pooling is then used to perform the subsequent classification task, completing the fine-tuning phase under supervision. After training, only the trained encoder module is retained. The decoder module is discarded, and the encoder output is pooled and then classified to complete the liveness detection task.

[0111] The embodiment of the present application first uses a mask image with partially occluded areas to train the feature extraction capability of the encoder, and then uses an unoccluded sample image to train the recognition and classification capability of the classifier. The two-stage model training can improve the convergence speed of the encoder and classifier, thereby improving the training efficiency of the model.

[0112] In one embodiment of the present application, in order to obtain an encoder that can encode facial features well, the model can be pre-trained by supervised comparison using live faces and attack images. The sample images used to train the liveness detection model in the embodiment of the present application may include live sample images corresponding to live objects and non-live sample images corresponding to non-live objects. Among them, the live sample images can be, for example, real face images obtained by face collection of real users, and the non-live sample images can be, for example, false face images forged or altered to imitate live sample images. Non-live sample images can also be called attack samples, for example, they can be face images forged through attack methods such as video playback and paper printing.

[0113] For example, the embodiments of the present application can use data sets such as Oulu-NPU, Replay-Attack, MSU-MFSD and CASIA-FASD to pre-train the model. These data sets include attack methods such as video playback and paper printing to forge faces.

[0114] FIG4 illustrates a method for training a liveness detection model based on contrastive learning in one embodiment of the present application. The training method can be performed independently by the terminal device or server shown in FIG1 , or by both the terminal device and the server. The present embodiment of the application uses the training method performed by the server as an example. As shown in FIG4 , the training method for the liveness detection model can also include the following steps S410 to S440.

[0115] S410: Combining two sample images into a sample pair, where the sample pair includes a positive sample pair consisting of two living sample images or a negative sample pair consisting of one living sample image and one non-living sample image.

[0116] S420: Compare the encoding features of the two sample images in the sample pair to obtain the feature similarity of the sample pair.

[0117] S430: Determine a contrast loss value of the sample image according to the feature similarity, where the contrast loss value is used to represent the ability of the encoder to extract similar coding features for multiple sample images corresponding to the living object.

[0118] S440: Update the model parameters of the encoder according to the contrast loss value.

[0119] The embodiment of the present application combines supervised contrastive learning and image mask pre-training methods to narrow the common features between living samples to find common information between living samples, further improving the generalization ability of the network and the test accuracy of unknown domains and unknown living attack types, and solving the problem of poor generalization against new attacks and cross-domains.

[0120] In one embodiment of the present application, after updating the model parameters of the encoder and decoder based on the reconstruction loss value between the output image and the sample image, the timing of introducing contrastive learning can be determined based on the degree of convergence of the encoder. For example, an error threshold value is obtained to represent the degree of convergence of the encoder, and when the reconstruction loss value is less than the preset error threshold value, the model parameters of the encoder are updated based on the contrastive loss value. In one embodiment, the error threshold value can be set based on experience or during training based on the training situation. In some cases, the contrastive loss value can also be referred to as the contrastive loss error.

[0121] In the embodiment of the present application, contrastive learning is introduced after the encoder converges to a certain degree, which can ensure that the encoding features used in contrastive learning are deep features with content significance extracted from the sample images, thereby improving the overall training speed of the model.

[0122] In other optional implementations, embodiments of the present application may alternate between mask reconstruction and contrastive learning training. For example, after the encoder has iterated through several rounds of mask reconstruction training and achieved convergence, contrastive learning is introduced; and after the encoder has iterated through several rounds of contrastive learning training and achieved convergence, mask reconstruction training is continued. By alternating between mask reconstruction and contrastive learning, both the model's accuracy in liveness detection and its generalization ability can be balanced, and the model training speed can be increased.

[0123] In one embodiment of the present application, a method for determining a contrast loss value of a sample image based on feature similarity may include: assigning different weight coefficients to positive sample pairs and negative sample pairs, respectively; and determining a contrast loss value of the sample image based on the weight coefficient and feature similarity.

[0124] In the embodiment of the present application, weight coefficients are assigned to different sample pairs according to the sample type, and the weight coefficients can be used to control the degree of influence of positive sample pairs or negative sample pairs on model parameters during model training.

[0125] In one embodiment of the present application, the weight coefficient of the positive sample pair is greater than the weight coefficient of the negative sample pair. Assigning a larger weight coefficient to the positive sample pair can shorten the distance between the living sample images, explore similar features between the living sample images, and improve the model's recognition ability for living sample images.

[0126] In some optional implementations, embodiments of the present application can dynamically assign weight coefficients to positive sample pairs and negative sample pairs. For example, the same weight coefficient can be assigned to positive sample pairs and negative sample pairs at the beginning of training. As the number of training iterations increases, the weight coefficient of the positive sample pairs can be gradually increased, or the weight coefficient of the negative sample pairs can be gradually decreased. By dynamically controlling the weight coefficients of the positive sample pairs and the negative sample pairs, the focus of training can be adjusted at different training stages, thereby balancing training efficiency and training reliability.

[0127] In one embodiment of the present application, the positive sample pairs include same-domain positive sample pairs and non-same-domain positive sample pairs. The same-domain positive sample pairs are composed of two living image samples with the same domain label, and the non-same-domain positive sample pairs are composed of two living image samples with different domain labels. The domain label is used to indicate the category information of the living image sample; the weight coefficient of the non-same-domain positive sample pairs is greater than the weight coefficient of the same-domain positive sample pairs.

[0128] For example, domain labels can include category information describing the image source or content scene. Image sources can include different types of image acquisition devices, such as smartphones, cameras, and camcorders. Content scenes can include environmental features of the scene in which the image was captured, such as indoor scenes, outdoor scenes, static backgrounds, and dynamic backgrounds.

[0129] The embodiments of the present application assign different weight coefficients to the same-domain positive sample pairs and the non-same-domain positive sample pairs, so as to control the influence of the same-domain positive sample pairs and the non-same-domain positive sample pairs on the model parameters during model training, thereby improving the flexibility of model training.

[0130] In some optional implementations, embodiments of the present application can dynamically assign weight coefficients to pairs of positive samples from the same domain and pairs of positive samples from different domains. For example, at the beginning of training, the same weight coefficients can be assigned to pairs of positive samples from the same domain and pairs of positive samples from different domains. As the number of training iterations increases, the weight coefficients of pairs of positive samples from the same domain can be gradually increased, or the weight coefficients of pairs of positive samples from different domains can be gradually decreased. By dynamically controlling the weight coefficients of pairs of positive samples from the same domain and pairs of positive samples from different domains, the focus of training can be adjusted at different training stages, thereby balancing the reliability training and generalization training of the model.

[0131] In one embodiment of the present application, blocking a portion of the sample image in step S210 to obtain a mask image including a non-masked area may further include: dividing the sample image into a plurality of tiles of the same size; randomly blocking a number of tiles in the sample image to obtain a mask image, the mask image including a mask area consisting of the blocked tiles and a non-masked area consisting of the unblocked tiles.

[0132] The embodiment of the present application adopts an image segmentation method to randomly block several blocks of the same size, which can eliminate the influence of the size distribution of the mask area on the image reconstruction results during model training and improve the training efficiency and accuracy of the model.

[0133] In one embodiment of the present application, the number of tiles in the masked area of ​​a mask image is greater than the number of tiles in the unmasked area. By masking most of the content in the sample image, the embodiment of the present application can increase the degree of content loss between the mask image and the original sample image while retaining the core content of the image reconstruction, thereby maximizing the feature extraction capability of the encoder and improving the training efficiency of the model.

[0134] In some optional implementations, the embodiments of the present application can dynamically adjust the number of tiles in the masked area and the non-masked area in the mask image during the training process. For example, the embodiments of the present application can block a smaller number of tiles at the beginning of the training so that the number of tiles in the masked area is smaller than the number of tiles in the non-masked area; as the number of iterative training increases, the number of tiles in the masked area is gradually increased, and the number of tiles in the non-masked area is reduced. By dynamically adjusting the number of tile distributions in the masked area, the embodiments of the present application can gradually increase the difficulty of feature extraction of the encoder while ensuring that the encoder converges as quickly as possible, thereby balancing the model training speed and training accuracy.

[0135] The embodiments of the present application are based on contrastive learning. The encoder output can be aggregated and used as input for supervised comparison. Both the target label and the domain label are required during the comparison phase. Samples are classified based on these two labels to obtain different sample categories. This is then used to bring samples of the same category closer together while samples of different categories are pushed further apart. Because contrastive learning is based on the aggregated features of the encoder's output, it needs to be introduced after the encoder has converged to a certain level. The timing of introducing contrastive learning can be determined based on changes in the reconstruction loss value.

[0136] In one embodiment of the present application, the contrast loss error may be calculated by a loss function, which may be a contrast loss function. The contrast loss function used to calculate the contrast loss value is shown below.

[0137] In the contrastive loss function, s i,j and s i,k represents the sample pair, y represents the domain label of the sample, τ is the preset temperature coefficient, λ l is the preset weight coefficient, and N represents the total number of sample pairs.

[0138] Positive samples can be divided into two categories: one is positive samples from the same domain, and the other is positive samples from different domains. Giving a larger weight coefficient λ to positive samples from different domains can further shorten the distance between positive samples from different domains. At the same time, we give smaller weights to negative samples, which can further enhance the impact of positive samples.

[0139] This embodiment overcomes the drawback of requiring a large amount of pre-training data for model training, achieving a significant performance improvement without requiring any additional datasets. Furthermore, the contrast loss function is specifically designed based on the classification of different labels, making this method more suitable for liveness detection tasks.

[0140] In one embodiment of the present application, the encoding order can be determined in advance for each block in the mask image, and then features can be extracted for each block according to the encoding order, that is, the position information of the block can be introduced in the feature extraction stage of the encoder and the image restoration stage of the decoder.

[0141] On this basis, obtaining the encoding features of the non-masked area obtained by the encoder through feature extraction of the non-masked area may further include: performing linear mapping on each tile in the non-masked area to obtain a semantic feature vector for the tile; determining the encoding order of the tile based on the distribution position of the tile in the sample image; obtaining the encoding features of the tile obtained by the encoder through feature extraction of the semantic feature vector of each tile; and concatenating the encoding features of each tile into the encoding features of the non-masked area according to the encoding order of the tile. In the ViT encoder, linear mapping is implemented by a linear layer (fully connected layer) to map the input features to a new feature space. Linear mapping plays a key role in multiple steps of image processing, including image tile embedding, generation of query / key / value vectors, and feature transformation in feedforward neural networks. The core of linear mapping is linear transformation achieved through matrix multiplication and addition. In some embodiments, the encoding order of the tiles can be a pre-set order that remains consistent during the encoder and decoder training process and / or testing and use. This encoding order can be, for example, from left to right and from top to bottom of the image. For example, in the schematic diagram of FIG7 , for the non-masked area, the encoding order is from left to right and from top to bottom, namely: tile a, tile c, tile d, tile h, and tile i.

[0142] Obtaining an output image obtained by the decoder performing feature restoration on the coding features of the non-masked area may further include: determining the coding order of the blocks according to the distribution positions of the blocks in the sample image; splicing the coding features of the non-masked area and the coding features of the masked area according to the coding order and inputting the spliced ​​features into the decoder; obtaining an output image obtained by the decoder performing feature restoration on the coding features of the non-masked area and the coding features of the masked area.

[0143] The coding order of the tiles may be sequence information formed by sorting the distribution positions according to a preset arrangement direction. For example, the arrangement order of the distribution positions may be determined in order from left to right and from top to bottom.

[0144] FIG5 illustrates a method for training a liveness detection model based on encoding sequence in one embodiment of the present application. The training method can be performed independently by the terminal device or server shown in FIG1 , or by both the terminal device and the server. The present embodiment of the present application uses the training method performed by the server as an example. As shown in FIG5 , the training method for the liveness detection model can further include the following steps S501 to S512.

[0145] S501: masking a portion of the sample image to obtain a mask image including a non-masked area, where the non-masked area is an unmasked area in the mask image.

[0146] S502: Perform linear mapping on each image block in the non-mask area to obtain a semantic feature vector of the image block.

[0147] S503: Determine the coding order of the image blocks according to the distribution positions of the image blocks in the sample image.

[0148] S504: Obtain coding features of the image blocks obtained by extracting features from the semantic feature vectors of each image block by the encoder.

[0149] S505: According to the coding order of the tiles, the coding features of the tiles are concatenated to form the coding features of the non-masked area.

[0150] S506: splicing the coding features of the non-masked area and the coding features of the masked area according to the coding order of the tiles and inputting the concatenated features into the decoder.

[0151] S507: Obtain an output image obtained by restoring the encoding features of the non-masked area and the encoding features of the masked area by the decoder.

[0152] S508: Update the model parameters of the encoder and decoder according to the reconstruction loss value between the output image and the sample image.

[0153] S509: Combining the coded features of the non-masked areas of the two sample images into a sample pair.

[0154] S510: Compare the encoding features of the non-masked areas of the two sample images to obtain feature similarity of the sample pairs.

[0155] S511: Determine the contrast loss value of the sample image based on the feature similarity.

[0156] S512: Update the model parameters of the encoder according to the contrast loss value.

[0157] For the specific implementation of each method step in the embodiments of the present application, reference can be made to the relevant contents of other embodiments mentioned above and will not be repeated here.

[0158] In one embodiment of the present application, a position feature vector can be determined in advance for each tile in the non-mask area of ​​the mask image, and then the position feature can be extracted while extracting the semantic features of the non-mask area, and the position information of the tile can be introduced in the feature extraction stage of the encoder or the image restoration stage of the decoder.

[0159] On this basis, obtaining the coding features of the non-masked area obtained by the encoder performing feature extraction on the non-masked area can further include: linearly mapping each block in the non-masked area to obtain a semantic feature vector of the non-masked area; determining the position feature vector of the non-masked area according to the distribution position of the block in the sample image; and obtaining the coding features of the non-masked area obtained by the encoder performing feature extraction on the semantic feature vector and the position feature vector of the non-masked area.

[0160] Obtaining an output image obtained by performing feature restoration on the coding features of the non-masked area by the decoder may further include: determining the position feature vector of the non-masked area based on the distribution position of the block in the sample image; and obtaining an output image obtained by performing feature restoration on the coding features of the non-masked area and the position feature vector of the non-masked area by the decoder.

[0161] In the embodiment of the present application, the position feature vector can have the same vector length as the semantic feature vector. When extracting features from the semantic feature vector and the position feature vector, the semantic feature vector and the position feature vector can be fused into one vector by vector addition, thereby fusing the semantic information and position information of the tile without changing the vector length.

[0162] Figure 6 shows a method for training a liveness detection model based on tile position features in an application scenario of the present application. The training method can be performed independently by the terminal device or server shown in Figure 1, or by the terminal device and the server together. The embodiment of the present application uses the training method performed by the server as an example. As shown in Figure 6, the training method of the liveness detection model can also include the following steps S601 to S610.

[0163] S601: masking a part of the sample image to obtain a mask image including a non-masked area, where the non-masked area is an unmasked area in the mask image.

[0164] S602: Perform linear mapping on each image block in the non-mask area to obtain a semantic feature vector of the non-mask area.

[0165] S603: Determine a position feature vector of the non-mask area according to the distribution position of the image blocks in the sample image.

[0166] S604: Obtain coding features of the non-masked area obtained by extracting the semantic feature vector and the position feature vector of the non-masked area by the encoder.

[0167] S605: Obtain an output image obtained by performing feature restoration on the coding features of the non-masked area and the position feature vector of the non-masked area by the decoder.

[0168] S606: Update the model parameters of the encoder and decoder according to the reconstruction loss value between the output image and the sample image.

[0169] S607: Combining the coding features of the non-masked areas of the two sample images into a sample pair.

[0170] S608: Compare the encoding features of the non-masked areas of the two sample images to obtain feature similarity of the sample pairs.

[0171] S609: Determine the contrast loss value of the sample image according to the feature similarity.

[0172] S610: Update the model parameters of the encoder according to the contrast loss value.

[0173] For the specific implementation of each method step in the embodiments of the present application, reference can be made to the relevant contents of other embodiments mentioned above and will not be repeated here.

[0174] FIG7 shows a schematic diagram showing the principle of training a liveness detection model in an application scenario according to an embodiment of the present application.

[0175] As shown in Figure 7, during the first phase of network pre-training, a sample image 701, serving as training data, is partially randomly masked, resulting in a masked image 702 with masked and unmasked regions. For example, the sample image 701 shown in the figure includes nine tiles, a through i. Tiles b, e, f, and g are randomly masked to form masked image 702. The unmasked region of masked image 702 includes the remaining unmasked tiles a, c, d, h, and i.

[0176] After linear mapping of the unobstructed blocks a, c, d, h, and i, a semantic feature vector 703 is obtained. After the encoder extracts features from the semantic feature vector 703, the encoding features V1, V3, V4, V8, and V9 of the non-masked area are obtained, which is the area 704 in FIG7 .

[0177] The coded features V1, V3, V4, V8 and V9 output by the encoder, and the coded features M2, M5, M6 and M7 of the blocked mask area are arranged and spliced ​​according to the distribution positions of the tiles to obtain the coded features 705 of the mask image.

[0178] The encoded features 705 of the mask image are input to the decoder, and the decoder reconstructs the features of the original image of each block based on the input encoded features 705. The decoder can reconstruct each pixel of the original image to obtain an output image (i.e., reconstructed image 706).

[0179] After the loss function in the first phase converges to the specified threshold, network pre-training enters the second phase, which adds an additional contrastive learning algorithm. First, the features of each tile output by the encoder are concatenated and aggregated to obtain a representative token for each sample. This is then distinguished based on the domain label and liveness label, shortening the distance between live samples 707 and simultaneously increasing the distance between live samples 707 and attack samples 708. Different weights are assigned to samples of different categories in the loss function for the second phase of pre-training. The loss at this point is a linear combination of contrastive loss and reconstruction loss. This design ensures that the features extracted by the decoder are meaningful when used for comparison, rather than being random at the outset.

[0180] It should be noted that although the steps of the method of the present application are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all steps must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0181] The following describes an embodiment of a device of the present application, which can be used to execute the training method of the liveness detection model in the above-mentioned embodiment of the present application. FIG8 schematically shows a block diagram of the structure of the training device of the liveness detection model provided in the embodiment of the present application. As shown in FIG8, the training device 800 of the liveness detection model includes:

[0182] The masking module 810 is configured to mask a portion of the sample image to obtain a mask image including a non-masked area, where the non-masked area is an unmasked area in the mask image;

[0183] The encoding module 820 is configured to obtain encoding features of the non-masked area obtained by the encoder performing feature extraction on the non-masked area;

[0184] The decoding module 830 is configured to obtain an output image obtained by restoring the encoded features of the non-masked area by the decoder;

[0185] The updating module 840 is configured to update the model parameters of the encoder and the decoder according to the reconstruction loss value between the output image and the sample image.

[0186] In some embodiments of the present application, based on the above technical solution, the sample images include living sample images corresponding to living objects and non-living sample images corresponding to non-living objects; the training device 800 for the living body detection model further includes:

[0187] a sample pair acquisition module configured to combine the two sample images into a sample pair, wherein the sample pair includes a positive sample pair consisting of two living sample images or a negative sample pair consisting of one living sample image and one non-living sample image;

[0188] a feature comparison module, configured to compare the encoding features of the two sample images in the sample pair to obtain the feature similarity of the sample pair;

[0189] an error determination module, configured to determine a contrast loss value of the sample image based on the feature similarity;

[0190] A parameter updating module is configured to update the model parameters of the encoder according to the contrast loss value.

[0191] In some embodiments of the present application, based on the above technical solution, the parameter update module is further configured to: obtain an error threshold used to represent the degree of convergence of the encoder; when the reconstruction loss value is less than the error threshold, update the model parameters of the encoder according to the contrast loss value.

[0192] In some embodiments of the present application, based on the above technical solution, the error determination module is further configured to: assign different weight coefficients to the positive sample pairs and the negative sample pairs, respectively; and determine the contrast loss value of the sample image based on the weight coefficient and the feature similarity.

[0193] In some embodiments of the present application, based on the above technical solution, the weight coefficient of the positive sample pair is greater than the weight coefficient of the negative sample pair.

[0194] In some embodiments of the present application, based on the above technical solution, the positive sample pairs include same-domain positive sample pairs and non-same-domain positive sample pairs, the same-domain positive sample pairs are composed of two live image samples with the same domain label, and the non-same-domain positive sample pairs are composed of two live image samples with different domain labels, and the domain label is used to indicate the category information of the live image sample; the weight coefficient of the non-same-domain positive sample pair is greater than the weight coefficient of the same-domain positive sample pair.

[0195] In some embodiments of the present application, based on the above technical solution, the shielding module 810 includes:

[0196] An image segmentation module is configured to segment a sample image into a plurality of tiles of the same size;

[0197] The block masking module is configured to randomly mask a number of the blocks in the sample image to obtain a mask image, wherein the mask image includes a mask area consisting of the masked blocks and a non-mask area consisting of the unmasked blocks.

[0198] In some embodiments of the present application, based on the above technical solution, the number of tiles in the masked area is greater than the number of tiles in the non-masked area.

[0199] In some embodiments of the present application, based on the above technical solution, the encoding module 820 includes:

[0200] a first mapping module configured to perform linear mapping on each image block in the non-masked area to obtain a semantic feature vector of the image block;

[0201] A first order determination module is configured to determine the encoding order of the image blocks according to the distribution positions of the image blocks in the sample image;

[0202] A first feature extraction module is configured to obtain encoding features of the image blocks obtained by the encoder performing feature extraction on the semantic feature vectors of the respective image blocks;

[0203] The feature splicing module is configured to splice the coding features of each of the image blocks into the coding features of the non-masked area according to the coding order of the image blocks.

[0204] In some embodiments of the present application, based on the above technical solution, the encoding module 820 includes:

[0205] a second mapping module, configured to perform linear mapping on each image block in the non-mask area to obtain a semantic feature vector of the non-mask area;

[0206] A first position determination module is configured to determine a position feature vector of the non-mask area according to a distribution position of the image block in the sample image;

[0207] The second feature extraction module is configured to obtain the encoding features of the non-masked area obtained by the encoder performing feature extraction on the semantic feature vector of the non-masked area and the position feature vector of the non-masked area.

[0208] In some embodiments of the present application, based on the above technical solution, the decoding module 830 includes:

[0209] a second order determination module, configured to determine the encoding order of the image blocks according to the distribution positions of the image blocks in the sample image;

[0210] a feature input module configured to concatenate the coding features of the non-masked area and the coding features of the masked area according to the coding order and input the concatenated features into a decoder;

[0211] The first feature restoration module is configured to obtain an output image obtained by the decoder performing feature restoration on the coding features of the non-masked area and the coding features of the masked area.

[0212] In some embodiments of the present application, based on the above technical solution, the decoding module 830 includes:

[0213] a second position determination module, configured to determine a position feature vector of the non-mask area according to a distribution position of the image block in the sample image;

[0214] The second feature restoration module is configured to obtain an output image obtained by the decoder performing feature restoration on the encoding features of the non-masked area and the position feature vector of the non-masked area.

[0215] In some embodiments of the present application, based on the above technical solution, the liveness detection model further includes a classifier for identifying liveness images based on the coding features; the training device 800 of the liveness detection model further includes:

[0216] a third feature extraction module, configured to obtain encoding features of the sample image obtained by the encoder performing feature extraction on the unobstructed sample image;

[0217] a feature classification module configured to obtain a recognition result obtained by the classifier through classification processing of the coding feature, wherein the recognition result is used to indicate whether the sample image is a living body image;

[0218] A parameter updating module is configured to update the model parameters of the encoder and the classifier according to the recognition result.

[0219] The specific details of the training device for the liveness detection model provided in each embodiment of the present application have been described in detail in the corresponding method embodiments and will not be repeated here.

[0220] FIG9 schematically shows a block diagram of a computer system structure of an electronic device for implementing an embodiment of the present application.

[0221] It should be noted that the computer system 900 of the electronic device shown in FIG9 is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0222] As shown in Figure 9, the computer system 900 includes a central processing unit (CPU) 901, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 902 or the program loaded from the storage part 908 into the random access memory (RAM) 903. Various programs and data required for system operation are also stored in the random access memory 903. The CPU 901, the read-only memory 902, and the random access memory 903 are connected to each other via a bus 904. An input / output interface 905 (i.e., an I / O interface) is also connected to the bus 904.

[0223] The following components are connected to the input / output interface 905: an input section 906 including a keyboard, a mouse, and the like; an output section 907 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 908 including a hard disk and the like; and a communication section 909 including a network interface card such as a local area network card or a modem. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the input / output interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like, is installed in the drive 910 as needed so that a computer program read therefrom can be installed into the storage section 908 as needed.

[0224] In particular, according to an embodiment of the present application, the processes described in the various method flow charts can be implemented as computer software programs. For example, an embodiment of the present application includes a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for executing the methods shown in the flow charts. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 909 and / or installed from a removable medium 911. When the computer program is executed by the central processing unit 901, the various functions defined in the system of the present application are performed.

[0225] It should be noted that the computer-readable medium shown in the embodiments of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0226] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of the boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0227] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiment of the application, the features and functions of two or more modules or units described above can be concretized in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be concretized.

[0228] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the embodiments of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiments of the present application.

[0229] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of this application and include common knowledge or customary techniques in the art that are not disclosed herein.

[0230] It should be understood that the present application is not limited to the exact structures described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for training a liveness detection model, characterized in that: The liveness detection model includes an encoder for extracting coding features from an image, wherein the coding features are used to identify whether an object in the image is a live object; and the training method includes: Masking a portion of the sample image to obtain a mask image including a non-masked area, where the non-masked area is an unmasked area in the mask image; Obtaining a region coding feature obtained by extracting features of the non-masked region by the encoder; Obtaining an output image obtained by restoring the region coding features by a decoder; The model parameters of the encoder and the decoder are updated according to the reconstruction loss value between the output image and the sample image.

2. The training method of the liveness detection model according to claim 1, characterized in that: The sample images include living sample images corresponding to living objects and non-living sample images corresponding to non-living objects; after updating the model parameters of the encoder and the decoder according to the reconstruction loss value between the output image and the sample images, the method further includes: The two sample images are combined into a sample pair, wherein the sample pair includes a positive sample pair composed of two living sample images or a negative sample pair composed of one living sample image and one non-living sample image; Comparing the encoding features of the two sample images in the sample pair to obtain the feature similarity of the sample pair; Determining a contrast loss value of the sample image according to the feature similarity, the contrast loss value being used to indicate an ability of the encoder to extract similar coding features for a plurality of the sample images corresponding to a living object; A model parameter of the encoder is updated according to the contrast loss value.

3. The training method of the living body detection model according to claim 1 or 2, characterized in that: Updating a model parameter of the encoder according to the contrast loss value includes: Obtaining an error threshold corresponding to the encoder; When the reconstruction loss value is less than the error threshold, the model parameters of the encoder are updated according to the contrast loss value.

4. The training method of a living body detection model according to any one of claims 1 to 3, characterized in that: Determining the contrast loss value of the sample image according to the feature similarity includes: Assign different weight coefficients to the positive sample pair and the negative sample pair respectively; A contrast loss value of the sample image is determined according to the weight coefficient and the feature similarity.

5. The training method of a living body detection model according to any one of claims 1 to 4, characterized in that: The weight coefficient of the positive sample pair is greater than the weight coefficient of the negative sample pair.

6. The training method of a living body detection model according to any one of claims 1 to 5, characterized in that: The positive sample pairs include same-domain positive sample pairs and non-same-domain positive sample pairs, the same-domain positive sample pairs are composed of two living image samples with the same domain label, the non-same-domain positive sample pairs are composed of two living image samples with different domain labels, and the domain label is used to indicate category information of the living image samples; the weight coefficient of the non-same-domain positive sample pairs is greater than the weight coefficient of the same-domain positive sample pairs.

7. The method for training a liveness detection model according to any one of claims 1 to 6, characterized in that: Mask part of the sample image to obtain a mask image including the non-masked area, including: Divide the sample image into multiple tiles of the same size; A number of the image blocks are randomly blocked in the sample image to obtain a mask image, wherein the mask image includes a mask area composed of blocked image blocks and a non-mask area composed of unblocked image blocks.

8. The method for training a living body detection model according to any one of claims 1 to 7, characterized in that: The number of tiles in the masked area is greater than the number of tiles in the non-masked area.

9. The method for training a living body detection model according to any one of claims 1 to 8, characterized in that: Acquiring the region encoding feature obtained by extracting features of the non-masked region by the encoder, comprising: Performing linear mapping on each image block in the non-masked area to obtain a semantic feature vector of the image block; Determining the coding order of the image blocks according to the distribution positions of the image blocks in the sample image; Obtaining encoding features of the image blocks obtained by extracting semantic feature vectors of each of the image blocks by the encoder; According to the coding order of the image blocks, the coding features of the respective image blocks are concatenated to form the regional coding features.

10. The method for training a living body detection model according to any one of claims 1 to 8, characterized in that: Acquiring a region encoding feature obtained by extracting features of the non-masked region by the encoder, comprising: Performing linear mapping on each image block in the non-mask area to obtain a semantic feature vector of the non-mask area; Determining a position feature vector of the non-mask area according to a distribution position of the image block in the sample image; The region coding feature is obtained by extracting, by the encoder, the semantic feature vector of the non-masked region and the position feature vector of the non-masked region.

11. The method for training a living body detection model according to any one of claims 1 to 10, characterized in that: Obtaining an output image obtained by restoring the region coding features by the decoder, including: Determining the coding order of the image blocks according to the distribution positions of the image blocks in the sample image; splicing the region coding features and the mask region coding features according to the coding order and inputting the concatenated features into a decoder; An output image is obtained by restoring the encoding features of the region and the encoding features of the mask region by the decoder.

12. The method for training a living body detection model according to any one of claims 1 to 11, characterized in that: Obtaining an output image obtained by restoring the region coding features by the decoder, including: Determining a position feature vector of the non-mask area according to a distribution position of the image block in the sample image; An output image is obtained by restoring the region coding features and the position feature vector of the non-masked region by the decoder.

13. The method for training a liveness detection model according to any one of claims 1 to 12, characterized in that: The living body detection model also includes a classifier for identifying a living body image according to the encoding feature; after updating the model parameters of the encoder and the decoder according to the reconstruction loss value between the output image and the sample image, the method further includes: Obtaining encoding features of the sample image obtained by extracting features of the unobstructed sample image by the encoder; Acquire a recognition result obtained by the classifier classifying the coded features of the sample image, wherein the recognition result is used to indicate whether the sample image is a living body image; The model parameters of the encoder and the classifier are updated according to the recognition result.

14. A training device for a living body detection model, characterized in that: The liveness detection model comprises an encoder for extracting coding features from an image, wherein the coding features are used to identify whether an object in the image is a live object; and the training device comprises: A masking module is configured to mask a part of the sample image to obtain a mask image including a non-masked area, where the non-masked area is an unmasked area in the mask image; An encoding module, configured to obtain the region encoding feature obtained by the encoder performing feature extraction on the non-masked region; A decoding module is configured to obtain an output image obtained by restoring the regional coding features by a decoder; An updating module is configured to update the model parameters of the encoder and the decoder according to the reconstruction loss value between the output image and the sample image.

15. A computer readable medium, characterized in that The computer-readable medium stores a computer program, and when the computer program is executed by a processor, the training method of the living body detection model described in any one of claims 1 to 13 is implemented.

16. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; Wherein, the processor is configured to execute the executable instructions to implement the training method of the living body detection model as described in any one of claims 1 to 13.

17. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the training method of the living body detection model as described in any one of claims 1 to 13 is implemented.

Images