A method and device for training and using a living body detection model based on sample training

Abstract

The application discloses a kind of based on sample training's live body detection model training and use method, comprising: obtaining the entity model data and digital model data of target object;The entity model data and the digital model data are input into the first sample training network, model features are extracted by first feature extractor, the first discriminator is judged whether the model features belong to the digital model data or the entity model data, and the first training loss corresponding to the generative adversarial network formed by the first feature extractor and the first discriminator is calculated;With the digital model data and the entity model data as sample data, the live body judgment network is trained, and the judgment loss of the live body judgment network is calculated;According to the first training loss and the judgment loss, the live body detection model is trained.Correspondingly, the application discloses based on sample training's live body detection model training and use device.

Description

Technical Field

[0001] This invention relates to liveness detection technology, and more particularly to a method and apparatus for training and using a liveness detection model. Background Technology

[0002] With the widespread application of facial recognition technology in daily life, the threats it faces are increasing daily. Liveness detection is a crucial part of facial recognition systems; however, as 3D head models and masks become increasingly realistic, the quality of attacks on liveness detection is also improving, necessitating continuous optimization of existing liveness detection algorithms.

[0003] In optimizing liveness detection algorithms, the source of 3D head models has always been a challenge. Purchasing physical head models is not only expensive but also has a very long production cycle. Buying data acquisition services is costly but cannot guarantee data quality and availability, still failing to obtain sufficient data samples. Deep learning data augmentation methods such as random cropping, random rotation, adding blur noise, and color perturbation can only enrich the expression of samples at the lower-level information layer of the image, failing to fundamentally solve the problem of insufficient data samples. Therefore, new data generation methods are needed to provide sufficient data samples.

[0004] Therefore, we hope to obtain a new training scheme for liveness detection models to solve the problem of limited sample size. Summary of the Invention

[0005] One of the objectives of this invention is to provide a method for training and using a liveness detection model based on sample training. This method can generate sufficient data samples through digital model generation; and then introduce a feature alignment method to make full use of the data samples and improve the training and optimization efficiency of the liveness detection model.

[0006] In accordance with the aforementioned objective, this invention proposes a method for training a liveness detection model based on sample training. The liveness detection model includes a digital model network and a liveness judgment network. The digital model network includes a first sample training network, which in turn includes a first discriminator and a first feature extractor. The method includes:

[0007] Acquire the physical model data and digital model data of the target object;

[0008] The entity model data and the digital model data are input into the first sample training network. The first feature extractor extracts model features, and the first discriminator determines whether the model features belong to the digital model data or the entity model data. The first training loss corresponding to the generative adversarial network composed of the first feature extractor and the first discriminator is calculated.

[0009] The liveness detection network is trained using the digital model data and the entity model data as sample data, and the detection loss of the liveness detection network is calculated.

[0010] The liveness detection model is trained based on the first training loss and the judgment loss.

[0011] In this invention, entity model data and digital model data can be simultaneously input into the first sample training network in the digital model network to obtain the first training loss; then, the above two types of model data can be input into the liveness detection network to obtain the judgment loss; based on the obtained first training loss and judgment loss, the liveness detection model can be trained, so that the sample data is fully utilized and the efficiency and accuracy of liveness detection model training are greatly improved.

[0012] Further, training the liveness detection model based on the first training loss includes:

[0013] Simultaneously train the first discriminator and the first feature extractor;

[0014] In the generative adversarial network composed of the first feature extractor and the first discriminator, the goal is to minimize the discrimination loss, maximize the generation loss, and keep the discrimination loss of the first discriminator above a preset threshold, thereby training the liveness detection model.

[0015] Furthermore, after acquiring the entity model data and the digital model data, the method further includes:

[0016] The digital model data and the entity model data are fused to obtain fused model data;

[0017] The entity model data, the digital model data, and the fused model data are input into the first sample training network. The first feature extractor extracts model features, and the first discriminator determines whether the model features belong to the digital model data, the fused model data, or the entity model data. The first training loss is calculated by the generative adversarial network composed of the first feature extractor and the first discriminator.

[0018] Furthermore, the digital model network further includes a second sample training network, which includes a second discriminator and a second feature extractor. After obtaining the fused model data, the method further includes:

[0019] The digital model data and the fused model data are input into the second sample training network. The second feature extractor extracts model features, and the second discriminator determines whether the model features belong to the digital model data or the fused model data. The second training loss is calculated by the generative adversarial network composed of the second feature extractor and the second discriminator.

[0020] The digital model data, the fusion model data, and the entity model data are input into the first sample training network. The first feature extractor extracts model features, and the first discriminator determines whether the model features belong to the digital model data, the fusion model data, or the entity model data. The first training loss is calculated by the generative adversarial network composed of the first feature extractor and the first discriminator.

[0021] The liveness detection network is trained using the digital model data, the fusion model data, and the entity model data as sample data, and the detection loss of the liveness detection network is calculated.

[0022] The liveness detection model is trained based on the first training loss, the second training loss, and the judgment loss.

[0023] In this invention, by introducing data fusion to obtain fused model data, the distribution differences between digital model data and entity model data are narrowed in the image domain, so that the various attributes of the samples are more comprehensively represented and the quality of the samples is improved. By dividing the feature narrowing process into two stages, namely the second sample training and the first sample training, the computational complexity is reduced, feature alignment can be better achieved, and it is easy to use and can be easily combined with existing networks.

[0024] Furthermore, training the liveness detection model based on the first training loss and the second training loss includes:

[0025] Simultaneously train the second discriminator and the second feature extractor in the generative adversarial network composed of the second feature extractor and the second discriminator, with the goal of minimizing the discrimination loss and maximizing the generation loss, and keeping the discrimination loss of the second discriminator above a preset threshold, to train the liveness detection model;

[0026] Simultaneously train the first discriminator and the first feature extractor. In the generative adversarial network composed of the first feature extractor and the first discriminator, minimize the discrimination loss and maximize the generation loss, with the goal of keeping the discrimination loss of the first discriminator above a preset threshold, to train the liveness detection model.

[0027] Furthermore, the digital model data and the entity model data are fused to obtain fused model data, including:

[0028] The style transfer network was pre-trained on a general dataset.

[0029] The digital model data is used as content input, and the entity model data is used as style input. The input is fed into a pre-trained style transfer network to obtain the fusion model data.

[0030] Furthermore, acquiring the digital model data of the target object includes:

[0031] Collect multiple photos of the target object from different perspectives, and synthesize multiple frames to reconstruct the geometric information of the target object;

[0032] Measure the texture and material information of each region of the target object;

[0033] Ray tracing rendering is performed based on the geometric model, the texture material information, and the lighting information during the acquisition of the target object's photograph to obtain the rendered digital model data.

[0034] Furthermore, acquiring the digital model data of the target object includes:

[0035] A basic model is created based on the attribute characteristics of the target object;

[0036] Capture the geometric texture of the target object and build a fine-grained model based on the geometric texture;

[0037] Based on the base model and the fine-grained model, the digital model data is generated using a single-frame image fitting method through differential rendering.

[0038] In accordance with the above-mentioned objective, the present invention also proposes a method for detecting liveness, comprising:

[0039] Acquire detection images;

[0040] The detected image is input into the liveness detection network in a pre-trained liveness detection model, wherein the liveness detection model is trained using the method described in any one of claims 1 to 8.

[0041] Another objective of this invention is to provide an apparatus for training and using a liveness detection model based on sample training. This apparatus can generate digital model data and fuse it with physical model data to provide sufficient training samples and improve the training and optimization efficiency of the liveness detection model.

[0042] In accordance with the aforementioned objective, this invention proposes a training device for a liveness detection model based on sample training, comprising:

[0043] The acquisition module is used to acquire physical model data and digital model data of the target object;

[0044] The first sample module inputs the entity model data and the digital model data into the first sample training network, extracts model features through the first feature extractor, determines whether the model features belong to the digital model data or the entity model data through the first discriminator, and calculates the first training loss corresponding to the generative adversarial network composed of the first feature extractor and the first discriminator.

[0045] The liveness detection module trains the liveness detection network using the digital model data and the entity model data as sample data, and calculates the detection loss of the liveness detection network.

[0046] The training module trains the liveness detection model based on the first training loss and the judgment loss.

[0047] Furthermore, the training module simultaneously trains the first discriminator and the first feature extractor. In the generative adversarial network composed of the first feature extractor and the first discriminator, the goal is to minimize the discrimination loss, maximize the generation loss, and keep the discrimination loss of the first discriminator above a preset threshold to train the liveness detection model.

[0048] Furthermore, the sample-based liveness detection model training device also includes:

[0049] The fusion module performs data fusion on the digital model data and the entity model data to obtain fused model data;

[0050] The first sample module is used to input the entity model data, the digital model data, and the fusion model data into the first sample training network, extract model features through the first feature extractor, determine whether the model features belong to the digital model data, the fusion model data, or the entity model data through the first discriminator, and calculate the corresponding first training loss of the generative adversarial network composed of the first feature extractor and the first discriminator.

[0051] Furthermore, the sample-based liveness detection model training device also includes:

[0052] The second sample module inputs the digital model data and the fused model data into the second sample training network, extracts model features through the second feature extractor, determines whether the model features belong to the digital model data or the fused model data through the second discriminator, and calculates the corresponding second training loss through the generative adversarial network composed of the second feature extractor and the second discriminator.

[0053] The liveness detection module trains the liveness detection network using the digital model data, the fusion model data, and the entity model data as sample data, and calculates the detection loss of the liveness detection network.

[0054] The training module trains the liveness detection model based on the first training loss, the second training loss, and the judgment loss.

[0055] Furthermore, the training module simultaneously trains the second discriminator and the second feature extractor. In the generative adversarial network composed of the second feature extractor and the second discriminator, the goal is to minimize the discrimination loss, maximize the generation loss, and keep the discrimination loss of the second discriminator above a preset threshold, thereby training the liveness detection model.

[0056] The training module also trains the first discriminator and the first feature extractor simultaneously. In the generative adversarial network composed of the first feature extractor and the first discriminator, the goal is to minimize the discrimination loss and maximize the generation loss, with the discrimination loss of the first discriminator remaining above a preset threshold, to train the liveness detection model.

[0057] Furthermore, the fusion module obtains a style transfer network by pre-training on a general dataset; the digital model data is used as content input, and the entity model data is used as style input, which are then input into the pre-trained style transfer network to obtain the fused model data.

[0058] Furthermore, the acquisition model acquires multi-view photos of the target object, and the geometric information of the target object is reconstructed by multi-frame synthesis.

[0059] Measure the texture and material information of each region of the target object.

[0060] Ray tracing rendering is performed based on the geometric model, the texture material information, and the lighting information during the acquisition of the target object photo to obtain the rendered digital model data.

[0061] Furthermore, the acquisition model is used to create a basic model based on the attribute characteristics of the target object.

[0062] Capture the geometric texture of the target object and build a fine-grained model based on the geometric texture.

[0063] Based on the base model and the fine-grained model, the digital model data is generated using a single-frame image fitting method through differential rendering.

[0064] In accordance with the above-mentioned objective, the present invention also provides a liveness detection device, comprising:

[0065] The acquisition module is used to acquire detection images;

[0066] The judgment module inputs the detected image into the liveness judgment network in the pre-trained liveness detection model, wherein the liveness detection model is trained using the sample-based liveness detection model training method described in this invention.

[0067] The present invention also provides a computer-readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed in a computer, it causes the computer to perform the steps of the sample-based liveness detection model training and usage method described in the present invention.

[0068] The present invention also provides a computing device, which includes a memory and a processor, characterized in that the memory stores executable code, and when the processor executes the executable code, it performs the steps in the sample-based liveness detection model training and usage method described in the present invention.

[0069] The method and apparatus for training and using a liveness detection model based on sample training described in this invention have the following beneficial effects:

[0070] Data can be generated through digital models, providing ample data samples, which greatly improves the data generation speed and reduces costs. Furthermore, the introduction of a feature alignment method makes full use of the sample data, significantly improving the efficiency and accuracy of liveness detection model training.

[0071] By introducing data fusion to obtain fused model data, the distribution differences between digital model data and entity model data in the image domain are narrowed, enabling a more comprehensive representation of various attributes of the samples and improving the quality of the samples. By dividing the feature fusion process into two stages, namely the second sample training and the first sample training, the computational complexity is reduced, feature alignment can be better achieved, and it is easy to use and can be easily combined with existing networks. Attached Figure Description

[0072] Figure 1 An exemplary flowchart of one implementation of the liveness detection model training method based on sample training according to the present invention is shown.

[0073] Figure 2 The diagram illustrates, exemplarily, a data fusion method in one embodiment of the liveness detection model training method based on sample training according to the present invention.

[0074] Figure 3 An exemplary flowchart of another implementation of the sample-based liveness detection model training method of the present invention is shown.

[0075] Figure 4 The diagram illustrates, in one embodiment, the feature reduction process of the liveness detection model training method based on sample training according to the present invention.

[0076] Figure 5 An exemplary schematic diagram of the liveness detection model training device based on sample training according to the present invention is shown in one embodiment. Detailed Implementation

[0077] The training and use method and apparatus for a liveness detection model based on sample training according to the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. However, this detailed description does not constitute a limitation of the present invention.

[0078] The liveness detection mentioned in this invention includes, but is not limited to, methods that extract and judge biological features through image recognition, such as liveness detection of human faces and heads. It also includes liveness detection of human limb movements and liveness detection in the animal kingdom. The following embodiments primarily use liveness detection of human faces and heads as examples to facilitate the explanation of the solution. However, the methods and apparatus provided by this invention are not limited to the field of human liveness detection, and the following embodiments are not intended to limit the application scenarios of this solution.

[0079] In one embodiment of the present invention, a method for training a liveness detection model based on sample training is proposed. Figure 1 An exemplary flowchart of one implementation of the sample-based liveness detection model training method of the present invention is shown. Figure 1 As shown, the liveness detection model includes a digital model network and a liveness judgment network. The digital model network includes a first sample training network, which in turn includes a first discriminator and a first feature extractor. The specific methods include:

[0080] 100: Obtain the physical model data and digital model data of the target object.

[0081] Physical model data of human faces and heads can be obtained by ordering handcrafted physical head molds from head mold makers.

[0082] The digital model data is obtained using digital simulation methods, enabling the generation of a large amount of high-quality model data as data samples in a short time with limited cost. Compared to deep learning data augmentation methods such as random cropping, random rotation, adding blur noise, and color perturbation, the digital model data acquisition method used in this invention overcomes the limitation of only expressing the underlying information of images. For example, digital model data can be generated through computer image rendering and 3D face reconstruction. Computer graphics (CG) rendering technology can render digital model data after collecting information such as geometric information, texture material information, and lighting information of the face. 3D face reconstruction mainly utilizes deep learning technology, performing face modeling through a large training dataset, and introducing deep learning networks such as conditional StyleGAN, 3DDFA, and PRNet for more refined modeling. Combined with other rendering methods or neural networks, a reconstructed face view is fitted to obtain digital model data. The source of the digital model data is not limited; it can come from a pre-stored source database, or a data generation system can be added upstream of the liveness detection model, which provides model data using the aforementioned digital model data generation method.

[0083] 110: Input the entity model data and the digital model data into the first sample training network, extract the model features through the first feature extractor, determine whether the model features belong to the digital model data or the entity model data through the first discriminator, and calculate the first training loss corresponding to the generative adversarial network composed of the first feature extractor and the first discriminator.

[0084] The first sample training network, belonging to the digital model network in the liveness detection model of this invention, includes a first feature extractor and a first discriminator, and is a generative adversarial network (GAN). The first feature extractor extracts features from the model data, which can indicate whether the data leans more towards entity model data or digital model data. These features include, but are not limited to, texture information, material information, reflection, and geometric information, generating a generative loss in this process. The first discriminator determines whether the extracted features originate from entity model data or digital model data, generating a discriminative loss in this process, representing the degree of difference between the judgment result and the true result.

[0085] The first training loss can be a progressive adversarial loss, used to narrow the differences in feature distributions between different types of data in the feature domain. In this embodiment, it can be used to narrow the feature distributions of entity model data and digital model data, making the model data obtained through digital simulation increasingly closer to the entity model data, thereby achieving the purpose of supplementing the sample data volume with digital models. In this embodiment, the first training loss includes generation loss and discrimination loss.

[0086] 120: The liveness detection network was trained using digital model data and entity model data as sample data, and the detection loss of the liveness detection network was calculated.

[0087] After the feature approximation process in the first sample training network, the digital model data and the entity model data are used together as samples to input the liveness detection network. The output of this network is the liveness detection result, i.e., "is alive" or "is not alive".

[0088] The judgment loss represents the degree of difference between the judgment result of the liveness detection network and the actual result. In this invention, the labels of the model data are all known, that is, it is known before training whether the corresponding model data is "live" or "not live". The judgment loss can be obtained by comparing the output result of the liveness detection network with the corresponding label. The loss calculation method includes, but is not limited to, mean squared error, cross-entropy, etc.

[0089] 130: Train the liveness detection model based on the first training loss and the judgment loss.

[0090] The first training loss includes generation loss and discrimination loss. Training the model based on the first training loss can effectively improve the efficiency and quality of feature merging. The discrimination loss reflects the discrimination ability of the liveness detection network in the model. Training the model based on the discrimination loss is beneficial to improving the accuracy of liveness detection.

[0091] In the sample-based liveness detection model training scheme described in this invention, there are no restrictions on the type of liveness detection model; various models available in existing technologies can be applied to this scheme. Furthermore, the model data generation method and model training method described in this scheme can also be externalized to any existing model, demonstrating strong flexibility.

[0092] In some embodiments, training the liveness detection model based on a first training loss includes the steps of:

[0093] Simultaneously train the first discriminator and the first feature extractor. In the generative adversarial network composed of the first feature extractor and the first discriminator, the goal is to minimize the discrimination loss, maximize the generation loss, and keep the discrimination loss of the first discriminator above a preset threshold to train the liveness detection model.

[0094] The training process of a generative adversarial network is a process in which the generator, or in this embodiment, the "feature extractor," interacts with the discriminator. As the data generated by the generator becomes closer to the real data, the discriminator's judgment ability also becomes stronger. Therefore, in this embodiment, the first discriminator and the first feature extractor need to be trained simultaneously.

[0095] In this embodiment, ideally, the feature distributions of the digital model data and the physical model data are consistent, thus achieving the goal of obtaining data samples through digital simulation rather than custom-made physical head models. Therefore, the feature distribution of the digital model data extracted by the first feature extractor should be infinitely close to the feature distribution of the physical model data, so that the first discriminator will identify the digital model data as physical model data, achieving a "deceptive" effect. In this process, the generation loss needs to be maximized.

[0096] On the other hand, the discrimination loss of the first discriminator reflects its judgment ability, which means that it can determine that the input entity model data features belong to entity model data, and the input digital model data features can determine that the input digital model data belongs to digital model data. To ensure that the data model data can be "deceptively indistinguishable from the real thing," the first discriminator needs to be sufficiently reliable and its judgment results sufficiently accurate; therefore, the discrimination loss should be minimized. It should be noted that if the discrimination loss is infinitely small, it means that the first discriminator can always distinguish between digital model data and entity model data, and there will always be a difference between the two types of model data. However, as the feature distributions of digital model data and entity model data become increasingly similar, the first discriminator should find it increasingly difficult to distinguish them. At this point, a threshold needs to be set for the discrimination loss to keep it above the preset threshold, indicating that the first discriminator can no longer determine the source of the model data features, and the feature distributions of digital model data and entity model data have become sufficiently similar.

[0097] In some specific embodiments, training the liveness detection model based on the judgment loss may include: training the liveness judgment network with the goal of minimizing the judgment loss. Since accurate liveness judgment results are required, the liveness judgment network should possess strong judgment capabilities, thus minimizing the judgment loss is crucial.

[0098] In some specific embodiments, digital model data of the target object can be obtained through computer graphics (CG) rendering methods, including:

[0099] This method, also known as CG rendering, can generate a rendered camera view of a 3D digital head model. By capturing multi-view head photographs using a high-precision camera matrix, the original geometric information G of the head can be reconstructed using multi-frame synthesis. Then, equipment such as a colorimeter is used to measure the chromaticity, reflection, and other texture information T of various facial regions. Next, the baked normals can be segmented based on the UVs of the geometric model, and texture maps can be created based on the texture information. Finally, a camera imaging position is selected, and the lighting information captured at that head position is used for ray tracing and noise reduction post-processing rendering to obtain the rendered 3D digital head model camera view data X. cg .

[0100] In other embodiments, digital model data of the target object can also be obtained through three-dimensional face reconstruction:

[0101] This method can generate single-frame face reconstruction views and is the lowest-cost method for generating digital model data. First, it uses a large RGB-D dataset to learn attributes such as gender and age, and then builds a basic face model M. coarse Modeling; it can combine conditional StyleGAN and high-quality 3D scan data to capture details such as geometric textures and build fine-grained face models M. fine Based on the basic face model and the fine-grained face model, a large amount of face reconstruction view data X can be generated using differential rendering and single-image fitting. rec .

[0102] The RGB-D dataset is a dataset type that indicates each image in the dataset has two representations: a standard RGB three-channel color image and a depth image. Those skilled in the art can select specific datasets such as ScanNet or SUN RGB-D as needed.

[0103] In some other specific embodiments, methods such as 3DDFA and PRNet can also be used for 3D face reconstruction. 3DDFA is a face alignment method that uses unique PNCC feature encoding, representing depth information with color changes, and then fits a 3D face model through a CNN network. PRNet describes 3D information through a UV position map. For a 3D point cloud of a face, after projecting the model onto the xy plane, the x, y, and z coordinates of each pixel are saved in three images, similar to the R, G, and B channels, and then a CNN network is used for prediction.

[0104] In some embodiments, after acquiring the entity model data and the digital model data, the method further includes:

[0105] 200: Perform data fusion on digital model data and physical model data to obtain fused model data.

[0106] Because there are significant differences between digital model data and physical model data in the image domain, data fusion can be used to bridge this distributional gap. The fused data includes both new identity information provided by the digital model data, including but not limited to attributes such as age and gender, and information such as ambient lighting and underlying textures from the physical model data, thus narrowing the distributional differences between the digital and physical model data in the image domain.

[0107] Optionally, data fusion can be performed on digital model data and physical model data to obtain fused model data, as follows: Figure 2 As shown, Figure 2 The diagram illustrates, exemplarily, a data fusion method in one embodiment of the liveness detection model training method based on sample training according to the present invention, including:

[0108] The style transfer network was pre-trained on a general dataset.

[0109] The digital model data is used as the content input, and the entity model data is used as the style input. The input is fed into a pre-trained style transfer network to obtain the fused model data.

[0110] In some more specific embodiments, the data fusion method is expressed as follows:

[0111]

[0112] Among them, F ST For style transfer network functions; x s This represents the content input (i.e., the source), which in this embodiment is digital model data; x t The style input (i.e., the target) is, in this embodiment, entity model data; x a The fusion output (i.e., the enhanced target) represents the fusion model data in this embodiment; i, j, and k represent the sequence numbers of the corresponding model data.

[0113] In other, more specific embodiments, data fusion can also be performed using methods such as GAN or diffusion model.

[0114] 210: Input entity model data, digital model data and fusion model data into the first sample training network, extract model features through the first feature extractor, determine whether the model features belong to digital model data, fusion model data or entity model data through the first discriminator, and calculate the corresponding first training loss of the generative adversarial network composed of the first feature extractor and the first discriminator.

[0115] The first training loss can be the progressive adversarial loss, which is used to simultaneously narrow the distribution differences of the three data types—digital model data, fused model data, and entity model data—in the feature domain, i.e., feature distribution differences. In this embodiment, it includes generation loss and discrimination loss, so that the model data obtained through digital simulation and the fused data become closer and closer to the entity model data, in order to achieve the purpose of supplementing the amount of sample data by using digital models and data fusion.

[0116] In some specific embodiments, the digital model network further includes a second sample training network, which includes a second discriminator and a second feature extractor. Figure 3 An exemplary flowchart of another implementation of the sample-based liveness detection model training method of the present invention is shown. Figure 3 As shown, after obtaining the fusion model data, the following is also included:

[0117] 300: Input the digital model data and the fused model data into the second sample training network, extract the model features through the second feature extractor, determine whether the model features belong to the digital model data or the fused model data through the second discriminator, and calculate the corresponding second training loss of the generative adversarial network composed of the second feature extractor and the second discriminator.

[0118] The second sample training network also belongs to the digital model network in the liveness detection model of this invention. It includes a second feature extractor and a second discriminator, and is also a type of generative adversarial network. The second feature extractor can extract features from the model data, which can indicate whether the data is more biased towards digital model data or fused model data, generating a generative loss in this process. The second discriminator can determine whether the extracted features originate from digital model data or fused model data, generating a discriminative loss in this process.

[0119] The second training loss can also be an asymptotic adversarial loss, used to narrow the feature distributions of the digital model data and the fused model data, making the model data obtained through digital simulation and the model data after data fusion increasingly similar, thereby supplementing the amount of sample data. In this embodiment, the second training loss includes generation loss and discrimination loss.

[0120] Figure 4The illustration shows a schematic diagram of the feature reduction process in one implementation of the sample-based liveness detection model training method of the present invention. This process can also be called a progressive adversarial learning process, such as... Figure 4 The first step involves dealing with three different types of model data: digital model data, fused model data, and entity model data. This step first performs feature amplification on the digital model data and the fused model data, and obtains a set of these two types of model data after feature alignment. This can be regarded as a new model data type. This divides the feature amplification of the three types of model data into two stages. In each stage, feature amplification is performed between pairs of model data, which can reduce computational overhead and improve the efficiency of the overall feature amplification process.

[0121] 310: Input the digital model data, fusion model data and entity model data into the first sample training network, extract model features through the first feature extractor, determine whether the model features belong to the digital model data, fusion model data or entity model data through the first discriminator, and calculate the corresponding first training loss of the generative adversarial network composed of the first feature extractor and the first discriminator.

[0122] like Figure 4 The second step involves pairwise feature merging of the feature-aligned digital model data and fused model data with the entity model data. This progressive adversarial learning process breaks down feature merging between the three model data sets into two pairwise merging processes. This not only reduces computational complexity but also yields more accurate feature merging results. Compared to merging all three model data sets simultaneously, it achieves better feature alignment. Furthermore, the pairwise merging method is easier to use and can be readily integrated with existing networks.

[0123] 320: The liveness detection network is trained using digital model data, fusion model data, and entity model data as sample data, and the judgment loss of the liveness detection network is calculated.

[0124] After the feature-shortening process in the second and first sample training networks, the digital model data, fusion model data, and entity model data are used together as samples to input the liveness detection network. The output of this network is the liveness detection result, i.e., "is alive" or "is not alive".

[0125] The judgment loss represents the degree of difference between the judgment result of the liveness detection network and the actual result. In this invention, the labels of the model data are all known, that is, it is known before training whether the corresponding model data is "live" or "not live". The judgment loss can be obtained by comparing the output result of the liveness detection network with the corresponding label. The loss calculation method includes, but is not limited to, mean squared error, cross-entropy, etc.

[0126] 330: Train the liveness detection model based on the first training loss, the second training loss, and the judgment loss.

[0127] Both the first and second training losses include generation loss and discrimination loss. Training the model based on these two losses can effectively improve the efficiency and quality of feature merging. The discrimination loss reflects the discrimination ability of the liveness detection network in the model. Training the model based on this loss is beneficial to improving the accuracy of liveness detection.

[0128] In the sample-based liveness detection model training scheme described in this invention, there are no restrictions on the type of liveness detection model; various models available in existing technologies can be applied to this scheme. Furthermore, the model data generation method and model training method described in this scheme can also be externalized to any existing model, demonstrating strong flexibility.

[0129] In some specific embodiments, the liveness detection model is trained based on a first training loss and a second training loss, including:

[0130] Simultaneously train the second discriminator and the second feature extractor. In the generative adversarial network composed of the second feature extractor and the second discriminator, the goal is to minimize the discrimination loss, maximize the generation loss, and keep the discrimination loss of the second discriminator above a preset threshold to train the liveness detection model.

[0131] During the training process of a generative adversarial network, as the data generated by the generator becomes closer to real data, the discriminator's judgment ability also becomes stronger. Therefore, in this embodiment, the second discriminator and the second feature extractor need to be trained simultaneously.

[0132] In this embodiment, under ideal conditions, the feature distributions of the digital model data and the fused model data are consistent. Therefore, the feature distribution of the fused model data extracted by the second feature extractor should be infinitely close to the feature distribution of the digital model data. In this process, the generation loss needs to be maximized.

[0133] On the other hand, the discrimination loss of the second discriminator reflects its judgment ability. To ensure that both model data types can be used as data samples for the liveness detection model, the second discriminator needs to be sufficiently reliable and accurate in its judgment results; therefore, the discrimination loss should be minimized. It should be noted that if the discrimination loss is infinitely small, it means that the second discriminator can always distinguish between the digital model data and the fused model data, and there will always be a difference between the two types of model data. However, as the feature distributions of the digital model data and the fused model data become increasingly similar, the second discriminator should find it increasingly difficult to distinguish them. At this point, a threshold needs to be set for the discrimination loss to keep it above the preset threshold. This indicates that the second discriminator can no longer determine the source of the model data features, the feature distributions of the digital model data and the fused model data have become sufficiently similar, and they are fully aligned in the feature domain.

[0134] Simultaneously train the first discriminator and the first feature extractor. In the generative adversarial network composed of the first feature extractor and the first discriminator, the goal is to minimize the discrimination loss and maximize the generation loss, with the discrimination loss of the first discriminator remaining above a preset threshold, to train the liveness detection model.

[0135] In this embodiment, under ideal conditions, the feature distributions of digital model data, fused model data, and entity model data are consistent. Therefore, the feature distributions of the three model data extracted by the first feature extractor should be infinitely close. In this process, the generation loss needs to be maximized.

[0136] On the other hand, the discrimination loss of the first discriminator reflects its judgment ability. To ensure that all three model data types can serve as data samples for the liveness detection model, the first discriminator needs to be sufficiently reliable and accurate in its judgment results; therefore, the discrimination loss should be minimized. It should be noted that if the discrimination loss is infinitely small, it means that the first discriminator can always distinguish between digital model data, fused model data, and entity model data, and differences will always exist between these three types of model data. However, as the feature distributions of digital model data, fused model data, and entity model data become increasingly similar, the first discriminator should find it increasingly difficult to distinguish them. At this point, a threshold needs to be set for the discrimination loss to maintain it above the preset threshold, indicating that the first discriminator can no longer determine the source of the model data features, and the feature distributions of the three types of model data have become sufficiently similar.

[0137] In some specific embodiments, training the liveness detection model based on the judgment loss may include: training the liveness judgment network with the goal of minimizing the judgment loss. Since accurate liveness judgment results are required, the liveness judgment network should possess strong judgment capabilities, thus minimizing the judgment loss is crucial.

[0138] In some more specific embodiments, the liveness detection model training method based on sample training described in this invention includes the following two training steps after obtaining the fused model data:

[0139] In the first half of the training, the digital model data X t and fusion model data X a The second sample is input into the training network, and the model features are extracted by the second feature extractor G and then processed by the second discriminator D. ta To determine whether the model features belong to the numerical model data or the fused model data, the second training loss is calculated for the generative adversarial network (GAN) composed of the second feature extractor and the second discriminator. The second discriminator and the second feature extractor are trained simultaneously. In the GAN composed of the second feature extractor and the second discriminator, the goal is to minimize the discrimination loss and maximize the generation loss. The discrimination loss of the second discriminator is maintained at a preset threshold. D With the above as the goal, the liveness detection model is trained.

[0140] The process of training the liveness detection model based on the second training loss can be represented by Equation 1:

[0141]

[0142] Where x refers to a specific model data; The loss represents the feature convergence between the digital model data and the fused model data, i.e., the second training loss; E represents the mathematical expectation.

[0143] The training process in this step first narrows down the feature distributions of the digital model data and the fused model data. Optionally, the set of digital model data and fused model data with narrowed feature distributions can be denoted as X. c .

[0144] In the latter half of the training, the digital model data X t Fusion model data X a and entity model data X s The network is trained by inputting the first sample, and the model features are extracted by the first feature extractor G and then processed by the first discriminator D. cs To determine whether the model features belong to digital model data, fused model data, or entity model data, the first training loss is calculated for the generative adversarial network composed of the first feature extractor and the first discriminator. Simultaneously, the first discriminator and the first feature extractor are trained. Within the generative adversarial network, the goal is to minimize the discrimination loss and maximize the generation loss, while maintaining the discrimination loss of the first discriminator above a preset threshold, thus training the liveness detection model.

[0145] In this embodiment, the second feature extractor and the first feature extractor use the same network, so they are both represented by G.

[0146] The process of training the liveness detection model based on the first training loss can be represented by Equation 2:

[0147]

[0148] Among them, X c This refers to the collection of digital model data and fused model data with similar feature distributions; The loss represents the reduction in the gap between the set of digital model data and fused model data after feature distribution reduction and the entity model data, i.e., the first training loss.

[0149] Then, during the feature reduction process, the total loss of the digital model network... for:

[0150]

[0151] Finally, using digital model data X t Fusion model data X a and entity model data X s The liveness detection network is trained using sample data, and its decision loss is calculated. The liveness detection model is then trained with the goal of minimizing this decision loss.

[0152] In one embodiment of the present invention, a method for using a liveness detection model based on sample training is also proposed, namely, a liveness detection method. This method involves acquiring detection images and inputting them into the liveness detection network of a pre-trained liveness detection model. The liveness detection model is trained using the aforementioned sample-based liveness detection model training method. Because the liveness detection model undergoes a feature distribution convergence process during training, although only the liveness detection network is used, it is still constrained by factors in the second and first training networks. Therefore, the accuracy of this liveness detection method can be guaranteed regardless of the type of data source.

[0153] In another embodiment of the present invention, a training device for a liveness detection model based on sample training is proposed. Figure 5 An exemplary schematic diagram of the liveness detection model training device based on sample training according to one embodiment of the present invention is shown, such as... Figure 5 As shown, it includes:

[0154] The acquisition module is used to acquire physical model data and digital model data of the target object;

[0155] The first sample module inputs entity model data and digital model data into the first sample training network, extracts model features through the first feature extractor, determines whether the model features belong to digital model data or entity model data through the first discriminator, and calculates the first training loss corresponding to the generative adversarial network composed of the first feature extractor and the first discriminator.

[0156] The liveness detection module trains the liveness detection network using digital model data and entity model data as sample data, and calculates the detection loss of the liveness detection network.

[0157] The training module trains the liveness detection model based on the first training loss and the judgment loss.

[0158] The acquisition module and other modules can be implemented on the same entity or on different entities. In one implementation, after acquiring the face image to be detected, the face recognition terminal can call the built-in first sample module, liveness detection module, and training module to perform loss calculation and training; all modules operate within the gate. In other implementations, the face recognition terminal can also function solely as an acquisition device, transmitting the data to the cloud after obtaining the face image to be detected for further loss calculation and training.

[0159] In the data acquisition module, physical model data of faces and heads can be obtained by ordering handmade physical head molds from head mold makers.

[0160] The digital model data is obtained using digital simulation methods, enabling the generation of a large amount of high-quality model data as data samples in a short time with limited cost. Compared to deep learning data augmentation methods such as random cropping, random rotation, adding blur noise, and color perturbation, the digital model data acquisition method used in this invention overcomes the limitation of only expressing the underlying information of images. For example, digital model data can be generated through computer image rendering and 3D face reconstruction. Computer graphics (CG) rendering technology can render digital model data after collecting information such as geometric information, texture material information, and lighting information of the face. 3D face reconstruction mainly utilizes deep learning technology, performing face modeling through a large training dataset, and introducing deep learning networks such as conditional StyleGAN, 3DDFA, and PRNet for more refined modeling. Combined with other rendering methods or neural networks, a reconstructed face view is fitted to obtain digital model data. The source of the digital model data is not limited; it can come from a pre-stored source database, or a data generation system can be added upstream of the liveness detection model, which provides model data using the aforementioned digital model data generation method.

[0161] In the first sample module, the first sample training network belongs to the digital model network in the liveness detection model of this invention, including a first feature extractor and a first discriminator, and is a generative adversarial network. The first feature extractor can extract features from the model data. These features can indicate whether the data is more inclined towards entity model data or digital model data, including but not limited to texture information, material information, reflection, geometric information, etc., generating a generative loss in this process. The first discriminator can determine whether the extracted features originate from entity model data or digital model data, generating a discriminative loss in this process, representing the degree of difference between the judgment result and the true result.

[0162] The first training loss can be a progressive adversarial loss, used to narrow the differences in feature distributions between different types of data in the feature domain. In this embodiment, it can be used to narrow the feature distributions of entity model data and digital model data, making the model data obtained through digital simulation increasingly closer to the entity model data, thereby achieving the purpose of supplementing the sample data volume with digital models. In this embodiment, the first training loss includes generation loss and discrimination loss.

[0163] In the liveness detection module, after the feature approximation process in the first sample training network, the digital model data and the entity model data are used together as samples to input the liveness detection network. The output of this network is the liveness detection result, i.e., "is alive" or "is not alive".

[0164] The judgment loss represents the degree of difference between the judgment result of the liveness detection network and the actual result. In this invention, the labels of the model data are all known, that is, it is known before training whether the corresponding model data is "live" or "not live". The judgment loss can be obtained by comparing the output result of the liveness detection network with the corresponding label. The loss calculation method includes, but is not limited to, mean squared error, cross-entropy, etc.

[0165] In the training module, the first training loss includes generation loss and discrimination loss. Training the model based on the first training loss can effectively improve the efficiency and quality of feature merging. The discrimination loss reflects the discrimination ability of the liveness detection network in the model. Training the model based on it is beneficial to improve the accuracy of liveness detection.

[0166] In the sample-based liveness detection model training scheme described in this invention, there are no restrictions on the type of liveness detection model; various models available in existing technologies can be applied to this scheme. Furthermore, the model data generation method and model training method described in this scheme can also be externalized to any existing model, demonstrating strong flexibility.

[0167] In some specific embodiments, the training module includes:

[0168] Simultaneously train the first discriminator and the first feature extractor;

[0169] In the generative adversarial network consisting of the first feature extractor and the first discriminator, the goal is to minimize the discrimination loss, maximize the generation loss, and keep the discrimination loss of the first discriminator above a preset threshold to train the liveness detection model.

[0170] The training process of a generative adversarial network is a process in which the generator, or in this embodiment, the "feature extractor," interacts with the discriminator. As the data generated by the generator becomes closer to the real data, the discriminator's judgment ability also becomes stronger. Therefore, in this embodiment, the first discriminator and the first feature extractor need to be trained simultaneously.

[0171] In this embodiment, ideally, the feature distributions of the digital model data and the physical model data are consistent, thus achieving the goal of obtaining data samples through digital simulation rather than custom-made physical head models. Therefore, the feature distribution of the digital model data extracted by the first feature extractor should be infinitely close to the feature distribution of the physical model data, so that the first discriminator will identify the digital model data as physical model data, achieving a "deceptive" effect. In this process, the generation loss needs to be maximized.

[0172] On the other hand, the discrimination loss of the first discriminator reflects its judgment ability, which means that it can determine that the input entity model data features belong to entity model data, and the input digital model data features can determine that the input digital model data belongs to digital model data. To ensure that the data model data can be "deceptively indistinguishable from the real thing," the first discriminator needs to be sufficiently reliable and its judgment results sufficiently accurate; therefore, the discrimination loss should be minimized. It should be noted that if the discrimination loss is infinitely small, it means that the first discriminator can always distinguish between digital model data and entity model data, and there will always be a difference between the two types of model data. However, as the feature distributions of digital model data and entity model data become increasingly similar, the first discriminator should find it increasingly difficult to distinguish them. At this point, a threshold needs to be set for the discrimination loss to keep it above the preset threshold, indicating that the first discriminator can no longer determine the source of the model data features, and the feature distributions of digital model data and entity model data have become sufficiently similar.

[0173] In some specific embodiments, training the liveness detection model based on the judgment loss may include: training the liveness judgment network with the goal of minimizing the judgment loss. Since accurate liveness judgment results are required, the liveness judgment network should possess strong judgment capabilities, thus minimizing the judgment loss is crucial.

[0174] In some specific embodiments, digital model data of the target object can be obtained through computer graphics (CG) rendering methods, including:

[0175] This method, also known as CG rendering, can generate a rendered camera view of a 3D digital head model. By capturing multi-view head photographs using a high-precision camera matrix, the original geometric information G of the head can be reconstructed using multi-frame synthesis. Then, equipment such as a colorimeter is used to measure the chromaticity, reflection, and other texture information T of various facial regions. Next, the baked normals can be segmented based on the UVs of the geometric model, and texture maps can be created based on the texture information. Finally, a camera imaging position is selected, and the lighting information captured at that head position is used for ray tracing and noise reduction post-processing rendering to obtain the rendered 3D digital head model camera view data X. cg .

[0176] In other embodiments, digital model data of the target object can also be obtained through three-dimensional face reconstruction:

[0177] This method can generate single-frame face reconstruction views and is the lowest-cost method for generating digital model data. First, it uses a large RGB-D dataset to learn attributes such as gender and age, and then builds a basic face model M. coarse Modeling; it can combine conditional StyleGAN and high-quality 3D scan data to capture details such as geometric textures and build fine-grained face models M. fine Based on the basic face model and the fine-grained face model, a large amount of face reconstruction view data X can be generated using differential rendering and single-image fitting. rec .

[0178] The RGB-D dataset is a dataset type that indicates each image in the dataset has two representations: a standard RGB three-channel color image and a depth image. Those skilled in the art can select specific datasets such as ScanNet or SUN RGB-D as needed.

[0179] In some other specific embodiments, methods such as 3DDFA and PRNet can also be used for 3D face reconstruction. 3DDFA is a face alignment method that uses unique PNCC feature encoding, representing depth information with color changes, and then fits a 3D face model through a CNN network. PRNet describes 3D information through a UV position map. For a 3D point cloud of a face, after projecting the model onto the xy plane, the x, y, and z coordinates of each pixel are saved in three images, similar to the R, G, and B channels, and then a CNN network is used for prediction.

[0180] In some embodiments, the sample-based liveness detection model training apparatus further includes:

[0181] The fusion module performs data fusion between digital model data and physical model data to obtain fused model data.

[0182] The first sample module is used to input entity model data, digital model data, and fused model data into the first sample training network, extract model features through the first feature extractor, determine whether the model features belong to digital model data, fused model data, or entity model data through the first discriminator, and calculate the corresponding first training loss of the generative adversarial network composed of the first feature extractor and the first discriminator.

[0183] Because there are significant differences between digital model data and physical model data in the image domain, data fusion can be used to bridge this distributional gap. The fused data includes both new identity information provided by the digital model data, including but not limited to attributes such as age and gender, and information such as ambient lighting and underlying textures from the physical model data, thus narrowing the distributional differences between the digital and physical model data in the image domain.

[0184] Optionally, methods for fusing digital model data and entity model data to obtain fused model data include:

[0185] The fusion module obtains the style transfer network through pre-training on a general dataset;

[0186] The digital model data is used as the content input, and the entity model data is used as the style input. The input is fed into a pre-trained style transfer network to obtain the fused model data.

[0187] In some more specific embodiments, the data fusion method is expressed as follows:

[0188]

[0189] Among them, F ST For style transfer network functions; x sThis represents the content input (i.e., the source), which in this embodiment is digital model data; x t The style input (i.e., the target) is, in this embodiment, entity model data; x a The fusion output (i.e., the enhanced target) represents the fusion model data in this embodiment; i, j, and k represent the sequence numbers of the corresponding model data.

[0190] In other, more specific embodiments, data fusion can also be performed using methods such as GAN or diffusion model.

[0191] Optionally, the entities that the fusion module operates on are not limited. It can be located in the same electronic device as the acquisition module, and perform data fusion immediately after acquiring entity model data and digital model data; or it can be set up separately in a program in the cloud or in the background, or placed in a program with other modules, and achieve model training through continuous communication with the acquisition device.

[0192] The first training loss can be the progressive adversarial loss, which is used to simultaneously narrow the distribution differences of the three data types—digital model data, fused model data, and entity model data—in the feature domain, i.e., feature distribution differences. In this embodiment, it includes generation loss and discrimination loss, so that the model data obtained through digital simulation and the fused data become closer and closer to the entity model data, in order to achieve the purpose of supplementing the amount of sample data by using digital models and data fusion.

[0193] In some specific embodiments, the sample-based liveness detection model training device further includes:

[0194] The second sample module inputs digital model data and fusion model data into the second sample training network, extracts model features through the second feature extractor, determines whether the model features belong to digital model data or fusion model data through the second discriminator, and calculates the corresponding second training loss through the generative adversarial network composed of the second feature extractor and the second discriminator.

[0195] The liveness detection module trains the liveness detection network using digital model data, fusion model data, and entity model data as sample data, and calculates the detection loss of the liveness detection network.

[0196] The training module trains the liveness detection model based on the first training loss, the second training loss, and the judgment loss.

[0197] The second sample training network in the second sample module also belongs to the digital model network in the liveness detection model of this invention. It includes a second feature extractor and a second discriminator, and is also a type of generative adversarial network (GAN). The second feature extractor extracts features from the model data, which can indicate whether the data leans more towards digital model data or fused model data, generating a generative loss in this process. The second discriminator determines whether the extracted features originate from digital model data or fused model data, generating a discriminative loss in this process.

[0198] The second training loss can also be an asymptotic adversarial loss, used to narrow the feature distributions of the digital model data and the fused model data, making the model data obtained through digital simulation and the model data after data fusion increasingly similar, thereby supplementing the amount of sample data. In this embodiment, the second training loss includes generation loss and discrimination loss.

[0199] Faced with three different types of model data—digital model data, fused model data, and entity model data—this step first performs feature amplification on the digital model data and the fused model data, and obtains a set of these two types of model data after feature alignment. This can be regarded as a new model data type. In this way, feature amplification of the three types of model data is divided into two stages. In each stage, feature amplification is performed between pairs of model data, which can reduce computational overhead and improve the efficiency of the overall feature amplification process.

[0200] Next, the output of the second sample module is input into the first sample module. The set of digital model data and fused model data after feature alignment is then compared with the entity model data to perform feature alignment, which is also a pairwise alignment process. This progressive adversarial learning process breaks down the feature alignment between the three types of model data into two pairwise alignment processes. This not only reduces computational complexity but also makes the feature alignment results more accurate. Compared to aligning the three types of model data simultaneously, it can achieve feature alignment faster and better. Furthermore, the pairwise alignment method is easier to use and can be easily combined with existing networks.

[0201] After the feature-sampling process in the second and first sample modules, the liveness detection module takes digital model data, fusion model data and entity model data as samples and inputs them into the liveness detection network. The output of the network is the liveness detection result, i.e., "is alive" or "is not alive".

[0202] In the training module, both the first and second training losses include generation loss and discrimination loss. Training the model based on these two losses can effectively improve the efficiency and quality of feature merging. The discrimination loss reflects the discrimination ability of the liveness detection network in the model. Training the model based on this loss is beneficial to improving the accuracy of liveness detection.

[0203] In the sample-based liveness detection model training scheme described in this invention, there are no restrictions on the type of liveness detection model; various models available in existing technologies can be applied to this scheme. Furthermore, the model data generation method and model training method described in this scheme can also be externalized to any existing model, demonstrating strong flexibility.

[0204] In some more specific embodiments, the training module includes:

[0205] (1) The training module trains the second discriminator and the second feature extractor simultaneously. In the generative adversarial network composed of the second feature extractor and the second discriminator, the goal is to minimize the discrimination loss, maximize the generation loss, and keep the discrimination loss of the second discriminator above the preset threshold to train the liveness detection model.

[0206] In this embodiment, under ideal conditions, the feature distributions of the digital model data and the fused model data are consistent. Therefore, the feature distribution of the fused model data extracted by the second feature extractor should be infinitely close to the feature distribution of the digital model data. In this process, the generation loss needs to be maximized.

[0207] On the other hand, the discrimination loss of the second discriminator reflects its judgment ability. To ensure that both model data types can be used as data samples for the liveness detection model, the second discriminator needs to be sufficiently reliable and accurate in its judgment results; therefore, the discrimination loss should be minimized. It should be noted that if the discrimination loss is infinitely small, it means that the second discriminator can always distinguish between the digital model data and the fused model data, and there will always be a difference between the two types of model data. However, as the feature distributions of the digital model data and the fused model data become increasingly similar, the second discriminator should find it increasingly difficult to distinguish them. At this point, a threshold needs to be set for the discrimination loss to keep it above the preset threshold. This indicates that the second discriminator can no longer determine the source of the model data features, the feature distributions of the digital model data and the fused model data have become sufficiently similar, and they are fully aligned in the feature domain.

[0208] (2) The training module trains the first discriminator and the first feature extractor at the same time. In the generative adversarial network composed of the first feature extractor and the first discriminator, the goal is to minimize the discrimination loss and maximize the generation loss, with the discrimination loss of the first discriminator kept above the preset threshold, to train the liveness detection model.

[0209] In this embodiment, under ideal conditions, the feature distributions of digital model data, fused model data, and entity model data are consistent. Therefore, the feature distributions of the three model data extracted by the first feature extractor should be infinitely close. In this process, the generation loss needs to be maximized.

[0210] On the other hand, the discrimination loss of the first discriminator reflects its judgment ability. To ensure that all three model data types can serve as data samples for the liveness detection model, the first discriminator needs to be sufficiently reliable and accurate in its judgment results; therefore, the discrimination loss should be minimized. It should be noted that if the discrimination loss is infinitely small, it means that the first discriminator can always distinguish between digital model data, fused model data, and entity model data, and differences will always exist between these three types of model data. However, as the feature distributions of digital model data, fused model data, and entity model data become increasingly similar, the first discriminator should find it increasingly difficult to distinguish them. At this point, a threshold needs to be set for the discrimination loss to maintain it above the preset threshold, indicating that the first discriminator can no longer determine the source of the model data features, and the feature distributions of the three types of model data have become sufficiently similar.

[0211] In some specific embodiments, training the liveness detection model based on the judgment loss may include: training the liveness judgment network with the goal of minimizing the judgment loss. Since accurate liveness judgment results are required, the liveness judgment network should possess strong judgment capabilities, thus minimizing the judgment loss is crucial.

[0212] In another embodiment of the present invention, a device for using a liveness detection model based on sample training is also proposed, namely a liveness detection device. This device acquires detection images and inputs them into a pre-trained liveness detection model's liveness determination network. The liveness detection model is trained using the aforementioned sample-based liveness detection model training method. Because the liveness detection model undergoes a feature distribution convergence process during training, although only the liveness determination network is used, it is still constrained by factors in the second and first training networks. Therefore, the accuracy of this liveness detection method can be guaranteed regardless of the type of data source.

[0213] This invention employs a digital generation scheme to provide sufficient and high-quality data samples for training a liveness detection model, and uses generative adversarial networks to bring data features from different sources closer together, providing constraints for the training of the liveness detection model and ensuring the accuracy of liveness detection and the training efficiency of the model.

[0214] One embodiment of the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the above-described method for training and using a liveness detection model based on sample training provided by the present invention.

[0215] One embodiment of the present invention provides a computing device including a memory and a processor. The memory stores executable code. When the processor executes the executable code, it performs the above-described method for training and using a liveness detection model based on sample training provided by the present invention.

[0216] It should be noted that the above examples are merely specific embodiments of the present invention, and the present invention is obviously not limited to the above embodiments, with many similar variations. All modifications that can be directly derived or conceived by those skilled in the art from the content disclosed in this invention should fall within the protection scope of this invention.

Claims

1. A method for training a liveness detection model based on sample training, the liveness detection model comprising a digital model network and a liveness judgment network, the digital model network comprising a first sample training network, the first sample training network comprising a first discriminator and a first feature extractor, the method comprising: Obtain the physical model data of the target object and use digital simulation methods to obtain the digital model data of the target object; The entity model data and the digital model data are input into the first sample training network. The first feature extractor extracts model features, and the first discriminator determines whether the model features belong to the digital model data or the entity model data. The first training loss corresponding to the generative adversarial network composed of the first feature extractor and the first discriminator is calculated. The first loss function is a progressive adversarial loss function, which is used to bring the feature distributions of the entity model data and the digital model data closer together, so that the digital model data obtained through digital simulation becomes closer and closer to the entity model data. The liveness detection network is trained using the digital model data and the entity model data after feature amplification as sample data, and the detection loss of the liveness detection network is calculated. The liveness detection model is trained based on the first training loss and the judgment loss.

2. The liveness detection model training method based on sample training as described in claim 1, wherein the liveness detection model is trained according to the first training loss, comprising: Simultaneously train the first discriminator and the first feature extractor; In the generative adversarial network composed of the first feature extractor and the first discriminator, the goal is to minimize the discrimination loss, maximize the generation loss, and keep the discrimination loss of the first discriminator above a preset threshold, thereby training the liveness detection model.

3. The liveness detection model training method based on sample training as described in claim 1, after acquiring entity model data and digital model data, the method further includes: The digital model data and the entity model data are fused to obtain fused model data; The entity model data, the digital model data, and the fused model data are input into the first sample training network. The first feature extractor extracts model features, and the first discriminator determines whether the model features belong to the digital model data, the fused model data, or the entity model data. The first training loss is calculated by the generative adversarial network composed of the first feature extractor and the first discriminator.

4. The liveness detection model training method based on sample training as described in claim 3, wherein the digital model network further includes a second sample training network, the second sample training network including a second discriminator and a second feature extractor, and after obtaining the fused model data, the method further includes: The digital model data and the fused model data are input into the second sample training network. The second feature extractor extracts model features, and the second discriminator determines whether the model features belong to the digital model data or the fused model data. The second training loss is calculated by the generative adversarial network composed of the second feature extractor and the second discriminator. The digital model data, the fusion model data, and the entity model data are input into the first sample training network. The first feature extractor extracts model features, and the first discriminator determines whether the model features belong to the digital model data, the fusion model data, or the entity model data. The first training loss is calculated by the generative adversarial network composed of the first feature extractor and the first discriminator. The liveness detection network is trained using the digital model data, the fusion model data, and the entity model data as sample data, and the detection loss of the liveness detection network is calculated. The liveness detection model is trained based on the first training loss, the second training loss, and the judgment loss.

5. The liveness detection model training method based on sample training as described in claim 4, wherein the liveness detection model is trained according to the first training loss and the second training loss, comprising: Simultaneously train the second discriminator and the second feature extractor in the generative adversarial network composed of the second feature extractor and the second discriminator, with the goal of minimizing the discrimination loss and maximizing the generation loss, and keeping the discrimination loss of the second discriminator above a preset threshold, to train the liveness detection model; Simultaneously train the first discriminator and the first feature extractor in the generative adversarial network composed of the first feature extractor and the first discriminator, with the goal of minimizing the discrimination loss and maximizing the generation loss, and keeping the discrimination loss of the first discriminator above a preset threshold, to train the liveness detection model.

6. The liveness detection model training method based on sample training as described in claim 3, wherein data fusion is performed on the digital model data and the entity model data to obtain fused model data, including: The style transfer network was pre-trained on a general dataset. The digital model data is used as content input, and the entity model data is used as style input. The input is fed into a pre-trained style transfer network to obtain the fusion model data.

7. The liveness detection model training method based on sample training as described in claim 1, wherein acquiring the digital model data of the target object includes: Collect multiple photos of the target object from different perspectives, and synthesize multiple frames to reconstruct the geometric information of the target object; Measure the texture and material information of each region of the target object; Ray tracing rendering is performed based on the geometric model, the texture material information, and the lighting information during the acquisition of the target object's photograph to obtain the rendered digital model data.

8. The liveness detection model training method based on sample training as described in claim 1, wherein acquiring the digital model data of the target object includes: A basic model is created based on the attribute characteristics of the target object; Capture the geometric texture of the target object and build a fine-grained model based on the geometric texture; Based on the base model and the fine-grained model, the digital model data is generated using a single-frame image fitting method through differential rendering.

9. A method for detecting liveness, comprising: Acquire detection images; The detected image is input into the liveness detection network in a pre-trained liveness detection model, wherein the liveness detection model is trained using the method described in any one of claims 1 to 8.

10. A training device for a liveness detection model based on sample training, comprising: The acquisition module is used to acquire the physical model data of the target object and to acquire the digital model data of the target object using a digital simulation method; The first sample module inputs the entity model data and the digital model data into the first sample training network, extracts model features through a first feature extractor, determines whether the model features belong to the digital model data or the entity model data through a first discriminator, and calculates the first training loss corresponding to the generative adversarial network composed of the first feature extractor and the first discriminator. The first loss function is a progressive adversarial loss function, which is used to narrow down the feature distributions of the entity model data and the digital model data, so that the digital model data obtained through digital simulation becomes closer and closer to the entity model data. The liveness detection module trains the liveness detection network using the digital model data with approximated features and the entity model data as sample data, and calculates the detection loss of the liveness detection network. The training module trains the liveness detection model based on the first training loss and the judgment loss.

11. The liveness detection model training device based on sample training as described in claim 10, The training module trains the first discriminator and the first feature extractor simultaneously. In the generative adversarial network composed of the first feature extractor and the first discriminator, the goal is to minimize the discrimination loss, maximize the generation loss, and keep the discrimination loss of the first discriminator above a preset threshold, thereby training the liveness detection model.

12. The liveness detection model training apparatus based on sample training as described in claim 10, further comprising: The fusion module performs data fusion on the digital model data and the entity model data to obtain fused model data; The first sample module is used to input the entity model data, the digital model data, and the fusion model data into the first sample training network, extract model features through the first feature extractor, determine whether the model features belong to the digital model data, the fusion model data, or the entity model data through the first discriminator, and calculate the corresponding first training loss of the generative adversarial network composed of the first feature extractor and the first discriminator.

13. The liveness detection model training apparatus based on sample training as described in claim 12, further comprising: The second sample module inputs the digital model data and the fused model data into the second sample training network, extracts model features through the second feature extractor, determines whether the model features belong to the digital model data or the fused model data through the second discriminator, and calculates the corresponding second training loss through the generative adversarial network composed of the second feature extractor and the second discriminator. The liveness detection module trains the liveness detection network using the digital model data, the fusion model data, and the entity model data as sample data, and calculates the detection loss of the liveness detection network. The training module trains the liveness detection model based on the first training loss, the second training loss, and the judgment loss.

14. The liveness detection model training device based on sample training as described in claim 13, The training module trains the second discriminator and the second feature extractor simultaneously. In the generative adversarial network composed of the second feature extractor and the second discriminator, the goal is to minimize the discrimination loss and maximize the generation loss, while keeping the discrimination loss of the second discriminator above a preset threshold, in order to train the liveness detection model. The training module also trains the first discriminator and the first feature extractor simultaneously. In the generative adversarial network composed of the first feature extractor and the first discriminator, the goal is to minimize the discrimination loss and maximize the generation loss, with the discrimination loss of the first discriminator remaining above a preset threshold, to train the liveness detection model.

15. The liveness detection model training device based on sample training as described in claim 12, The fusion module obtains the style transfer network through pre-training on a general dataset; The digital model data is used as content input, and the entity model data is used as style input. The input is fed into a pre-trained style transfer network to obtain the fusion model data.

16. The liveness detection model training device based on sample training as described in claim 10, The acquisition model acquires multi-view photos of the target object and reconstructs the geometric information of the target object by synthesizing multiple frames. Measure the texture and material information of each region of the target object; Ray tracing rendering is performed based on the geometric model, the texture material information, and the lighting information during the acquisition of the target object photo to obtain the rendered digital model data.

17. The liveness detection model training device based on sample training as described in claim 10, The acquisition model is used to create a basic model based on the attribute characteristics of the target object; Capture the geometric texture of the target object and build a fine-grained model based on the geometric texture; Based on the base model and the fine-grained model, the digital model data is generated using a single-frame image fitting method through differential rendering.

18. A liveness detection device, comprising: The acquisition module is used to acquire detection images; The judgment module inputs the detected image into the liveness judgment network in a pre-trained liveness detection model, wherein the liveness detection model is trained using the method described in any one of claims 1 to 8.

19. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed in the computer, it causes the computer to perform the steps of the liveness detection model training method based on sample training as described in any one of claims 1-8.

20. A computing device comprising a memory and a processor, characterized in that, The memory stores executable code, and when the processor executes the executable code, it performs the steps in the liveness detection model training method based on sample training as described in any one of claims 1-8.

Images